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EXECUTIVE  SUMMARY 


Purpose 

This  report  documents  the  results  of  a  laboratory  investigation  of  the  B-2  radar  cursor 
and  its  effectiveness  in  terms  of  designation  speed  and  accuracy  as  well  as  user 
acceptance.  The  study  was  conducted  in  response  to  problems  crewmembers  are 
experiencing  with  the  current  cursor  on  the  B-2  radar  display.  In  particular,  the  design 
of  the  current  radar  cursor  (i.e.,  the  width  of  the  cursor  bars)  makes  it  difficult  to 
accomplish  fine  cursor  positioning  (i.e.,  accurate  placement  of  the  cursor  within  the 
desired  one  pixel  of  the  target.)  This  affects  the  accuracy  of  radar  updates  and  impacts 
the  precision  of  the  navigation  and  bombing  solutions.  The  purpose  of  this  study  was  to 
evaluate  the  current  cursor  and  alternative  cursor  designs  for  the  B-2  SAR  imagery 
application  and  to  recommend  an  alternative  design. 

Method 

The  experiment  evaluated  the  current  cursor  and  two  alternative  designs  in  terms  of 
designation  speed  and  accuracy.  Eighteen  subjects,  consisting  of  current  or  previous 
USAF  pilots  or  radar  navigators,  completed  experimental  trials  in  which  they  performed 
a  radar  update  procedure  (i.e.,  identified  and  designated  specified  aim  points).  This 
procedure  was  conducted  using  simulated  SAR  imagery  presented  on  a  multipurpose 
display  unit  (MDU)  within  the  Computer-Assisted  Procedures  Trainer  (CAPT)  resident  at 
Armstrong  Laboratory  at  Wright  Patterson  Air  Force  Base,  Dayton,  Ohio.  Each  subject 
performed  40  experimental  trials  or  designation  events  with  each  of  the  three  cursors. 
Designation  was  evaluated  in  terms  of  the  speed  and  accuracy  of  cursor  placement. 
After  completing  the  trials,  subjects  were  asked  to  respond  to  a  questionnaire 
addressing  the  following  areas:  (1)  cursor  effectiveness,  (2)  designation  technique 
(3)  mechanization  of  the  cursor  controller,  and  (4)  the  fidelity  of  the  part-task  simulation. 

Results 

The  results  indicated  that  one  of  the  alternative  cursor  designs  (Cursor  2)  elicited 
significantly  higher  designation  accuracy  (mean  error  =1.94  pixels)  than  the  current 
cursor  (Cursor  1 ,  mean  error  =  2.36  pixels)  or  the  second  alternative  cursor  (Cursor  3, 
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mean  error  =  2.28  pixels).  Designation  speed  did  not  vary  significantly  across  cursors, 
occurring  approximately  17  seconds  into  a  trial.  Examination  of  the  cursor  position  over 
the  course  of  the  trial  indicated  that,  when  performing  the  update  procedure  with  the 
current  cursor,  subjects  initially  moved  the  cursor  away  from  the  intended  designation 
point,  before  returning  to  the  aim  point  and  designating.  When  asked  why  they  used 
this  strategy  (i.e.,  pulling  the  cursor  off  of  the  area  of  interest),  subjects  stated  that  the 
current  cursor  obscured  important  cues  necessary  for  accurate  designation.  With  both 
alternative  cursors,  subjects  tended  to  move  directly  toward  the  intended  designation 
point.  In  addition,  analysis  of  cursor  position  over  time  indicated  that  subjects  exhibited 
difficulty  with  fine  positioning  of  all  three  cursors.  When  moving  the  cursor  toward  the 
intended  designation  point,  subjects  “overshot”  the  intended  point  an  average  of  4.8 
times  per  trial  across  cursors.  Subjects  attributed  this  overshooting  before  designation 
to  problems  associated  with  the  mechanization  of  the  cursor  controller  which  did  not 
vary  across  cursors. 

Subjective  data  and  comments  gathered  from  subjects  supported  these  findings. 
Overall,  subjects  rated  the  effectiveness  of  the  current  cursor  as  “somewhat  ineffective," 
whereas  alternative  Cursor  2  was  rated  as  “very  effective.”  When  asked  to  choose 
which  cursor  they  preferred,  10  subjects  chose  alternative  Cursor  2,  seven  chose 
alternative  Cursor  3,  and  one  chose  the  current  cursor  (Cursor  1). 

Conclusion 

The  improved  performance  with  Cursor  2  is  attributed  to  its  longer,  thinner  bars  as  well 
as  its  smaller  center  opening  and  its  perpendicular  terminators.  According  to  subjective 
comments,  these  factors  aided  in  determining  the  center  of  the  cursor  and  aligning  the 
cursor  over  the  intended  designation  point.  However,  the  cursor  controller 
mechanization,  which  was  not  addressed  in  this  study,  was  also  determined  to  be  a 
cause  of  poor  designation  strategy,  regardless  of  cursor  type. 
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SECTION  I 
Introduction 

This  section  of  the  report  presents  an  overview  of  the  background  and  approach  to  the 
experiment  as  well  as  a  discussion  of  how  the  alternative  cursors  were  developed. 

Background 

In  June  1991,  Armstrong  Laboratory  (AL/CFHI)  at  Wright  Patterson  Air  Force  Base, 
Dayton,  Ohio,  was  chartered  to  support  the  B-2  System  Program  Office  (SPO)  in  the 
development,  operation  and  enhancement  of  the  B-2  Cockpit  Evaluation  Facility  (CEF). 
The  CEF  consists  of  an  Engineering  Hotbench,  otherwise  known  as  the  Prototyping  and 
Evaluation  Station  (P&ES);  a  B-2  crewstation  simulator;  a  display  prototyping 
development  station  and  a  government-provided  secure  facility.  The  CEF  is  currently 
being  utilized  to  assess  current  and  future  B-2  aircraft  cre'vstation  design  approaches, 
allowing  researchers  and  developers  to  rapidly  respond  to  a  wide  range  of  B-2  control 
and  display-related  issues.  This  CEF  capability  affords  researchers  the  ability  to  develop 
prototype  Multipurpose  Display  Unit  (MDU)  displays,  evaluate  displays  in  a  semi- 
operational  environment,  and  provide  feedback  to  the  B-2  SPO.  The  B-2  CEF 
represents  a  unique  engineering  capability  that  has  been  specifically  designed  to  meet 
the  study  needs  associated  with  the  evaluation  of  advanced  controls  and  displays. 

Rapid  prototyping  software,  a  comprehensive  set  of  generic  aircraft  models,  and  a 
flexible  simulation  control  and  data  collection/reduction  capability  are  key  features. 

Within  the  context  of  this  B-2  support,  AL/CFHI  was  approached  by  the  B-2  SPO  who 
expressed  the  concern  that  B-2  crewmembers  were  experiencing  problems  with  the 
cursor  used  on  the  radar  display.  In  particular,  the  design  of  the  current  radar  cursor 
(i.e.,  three  pixels  in  width)  made  it  difficult  to  accomplish  fine  cursor  positioning 
(i.e.,  accurate  placement  of  the  cursor  within  the  desired  one  pixel  of  the  target.)  The 
B-2  crewmembers  had  little  confidence  in  the  accuracy  of  their  radar  update 
designations  and  were  concerned  about  the  subsequent  precision  of  the  navigation  and 
bombing  solutions. 

This  problem  was  briefed  at  the  15  April  1993  Watch  Item  Review  Board  and  upgraded 
to  Service  Report  Status.  Subsequently,  the  B-2  SPO  became  involved  and  asked 
AL/CFHI  at  WPAFB  to  investigate  other  possible  cursor  designs  for  this  application. 
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As  a  result  of  these  concerns,  AL/CFHI  was  tasked  to  conduct  a  laboratory  evaluation  of 
the  B-2  cursor  and  to  recommend  an  alternative  cursor  design  for  application  that  would 
allow  crewmembers  to  quickly  and  accurately  designate  aim  points  within  one  pixel. 
Specifically,  AL/CFHI  was  asked  to  arrive  at  an  optimum  size  and  shape  of  the  B-2 
radar  cursor. 

Approach 

A  review  of  published  literature  relevant  to  the  radar  cursor  study  was  conducted.  Of 
particular  interest  was  research  conducted  using  a  cursor  or  position  designator  to 
perform  a  target  designation  task  similar  to  that  required  by  the  B-2  Mission 
Commander.  Because  the  cursor  study  focused  on  target  designation  accuracy  and 
speed  for  a  SAR  imagery  application,  researchers  were  particularly  interested  in 
technical  reports  that  focused  on  similar  applications.  Online  computerized  databases 
(i.e.,  Defense  Technical  Information  Center  (DTIC)  Technical  Report  Bibliographic  (TR) 
Database  on  CD-ROM  and  DTIC  Work  Unit  Information  System  (WUIS)),  containing 
bibliographic  citations  to  over  650,000  technical  reports,  patent  application,  and 
conference  papers  covering  more  than  22  years,  were  searched  using  the  following 
keywords: 

•  control  dynamics 

•  cursor/radar  cursor 

•  position  designator 

•  control/display  gain  ratio 

•  isometric  control 

•  concave  force  controller 

The  search  for  published  work  in  the  area  of  cursor  size  and  shape  for  a  SAR  imagery 
application  or  a  similar  environment  yielded  very  little  relevant  research.  A  number  of 
reports  were  located  that  addressed  related  issues  such  as  relevant  factors  when 
selecting  input  devices  (joysticks,  touch  screen,  trackball,  etc.);  display/control  gain 
issues  (i.e.,  the  amount  of  movement  which  occurs  on  the  display  in  response  to  a  unit 
amount  of  movement  of  the  control);  and  isotonic  vs.  isometric  controls.  However, 
because  this  effort  was  limited  to  identifying  the  optimum  size  and  shape  of  the  radar 
cursor,  this  related  research  was  outside  the  scope  of  the  current  study. 
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A  decision  was  made  by  AL/CFHI  and  the  B-2  SPO  to  evaluate  the  effectiveness  of  the 
current  cursor  and  two  alternative  cursors.  The  task  upon  which  the  evaluation  was 
based  consisted  of  a  radar  update  procedure  in  which  subjects  identified  and 
designated  aim  points  using  the  current  and  two  alternative  cursors.  The  experimental 
approach  included  the  use  of  the  B-2  Computer- Assisted  Procedures  Trainer  (CAPT) 
resident  at  AL/CFHI  in  order  to  conduct  a  laboratory  evaluation  in  an  interactive,  semi- 
operational  environment.  The  procedure  was  conducted  using  simulated  SAR  imagery 
containing  representative  cultural  and  terrain  aim  points  displayed  at  the  CAPT’s 
Mission  Commander’s  Position.  A  decision  was  made  to  evaluate  the  cursors  on  two 
criteria,  including  the  speed  with  which  a  designation  was  made  and  the  accuracy  of 
designation.  In  addition  to  this  performance  data,  subjective  evaluations  of  the  cursors 
and  their  effectiveness  were  collected  from  the  subjects  participating  in  the  study. 

Cursor  Selection  And  Description 

Cursor  1  (i.e.,  the  current  cursor  used  on  the  B-2  radar  display)  and  its  dimensions  are 
shown  in  Figure  1.1.  This  cursor  is  composed  of  cursor  bars  that  are  three  pixels  wide 
by  ten  pixels  long,  and  has  a  center  opening  of  seven  by  seven  pixels.  As  mentioned 
above,  Cursor  1  was  described  by  B-2  aircrews  as  being  too  131"  because  the  wide 
bars  obscure  vital  visual  information  on  and  around  the  target.  Crewmembers  also 
commented  that  they  often  initially  slew  the  cursor  away  from  the  desired  designation 
point  in  order  to  clearly  see  the  area  of  interest.  The  design  of  the  alternative  cursors 
was  intended  to  eliminate  these  problems. 


1  _ 

iop  — H 


Cursor  1 


Figure  1.1.  Cursor  1:  Current  B-2  Synthetic  Aperture  Radar  Cursor. 
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The  design  of  alternative  cursors  involved  a  subjective  and  iterative  process.  Very  little 
previous  research  had  been  published  in  the  area  of  radar  cursor  design  in  terms  of  size 
and  shape.  Therefore,  the  design  process  focused  on  addressing  the  specific  concerns 
and  observations  of  the  B-2  crewmembers.  These  concerns  centered  around  the  width 
and  length  of  the  four  cursor  bars,  the  size  of  the  center  opening,  and  the  overall 
visibility  of  the  cursor.  The  first  and  most  obvious  alteration  to  the  current  cursor 
involved  designing  alternative  cursor  bars  that  were  thinner  and  did  not  obscure  as 
much  of  the  area  of  interest  on  the  SAR  imagery.  As  depicted  in  Figure  1.2,  both  of  the 
alternative  cursors  (i.e.  Cursors  2  and  3)  are  composed  of  bars  that  are  only  one  pixel 
wide. 


Cursor  2  Cursor  3 

Figure  1 .2.  Two  Alternative  Cursor  Designs  (Cursors  2  and  3). 


Once  the  cursors  were  reduced  in  width  (i.e.,  to  one  pixel  wide),  the  problem  of  image 
obscuration  was  diminished.  However,  the  thinness  of  the  bars  and  the  overall  small 
physical  size  of  the  cursors  made  it  much  less  conspicuous  and  very  difficult  to  see 
when  positioned  on  areas  of  the  imagery  that  contained  a  high  degree  of  background 
returns  (i.e.,  high-clutter  environments).  To  address  this  problem,  each  of  the  four 
cursor  bars  was  extended  in  length  from  10  pixels  (Cursor  1)  to  27  pixels  (Cursors  2 
and  3).  In  addition,  terminators  or  end  points  were  added  to  the  outside  end  of  each 
cursor  bar.  These  provide  visual  referents  to  facilitate  visual  localization  of  the  cursor 
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(especially  in  cluttered  imagery).  Additionally,  terminators  on  reticle  cursor  designs 
facilitate  vernier  alignment  processes  and  can  lead  to  greater  spatial  localization 
accuracies.  These  serve  as  an  aid  in  accomplishing  accurate  positioning  of  the  cursor. 
The  design  of  the  Cursor  2  terminators,  by  virtue  of  being  aligned  perpendicular  to  their 
respective  cursor  bars,  provides  an  additional  diagonal  vernier  referent  as  well.  To 
address  the  problem  of  the  wide  center  opening  in  the  current  cursor,  the  opening  of  the 
alternative  cursors  was  reduced  from  a  width  of  seven  pixels  to  five  pixels,  decreasing 
the  center  area  by  approximately  50%.  This  was  intended  to  reduce  the  amount  of 
visual  interpolation  required  to  estimate  the  exact  center  of  the  cursor  (see  Figure  1.2). 
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SECTION  II 
Method 


This  section  addresses  the  experimental  design  and  procedures  used  in  the  study. 

Subjects 

A  total  of  eighteen  (18)  subjects  participated  in  the  experiment.  As  depicted  in 
Table  2.1,  the  subjects  consisted  primarily  of  current  or  previous  USAF  operators  with 
radar  and  navigation  experience  in  the  B-2,  B-1,  and/or  B-52  aircraft.  Those  subjects 
that  had  experience  in  either  the  B-1  or  B-2  also  had  previous  experience  operating 
SAR. 


Table  2.1.  Subject  Background  and  Experience. 


Aircraft 

Type 

Position 

Sensor 

Experience 

Operating 

Crosshair 

B-2  =  2 

OSO  =  7 

SAR  =  9 

17 

00 

ii 

y — 

i 

£0 

WSO  =  2 

EVS/FLIR  =  14 

B-52  =  13 

RN  =  8 

Other  =  6 

NAV  =  7 

Pilot  =  1 

EWO  =  3 

Apparatus 

The  study  was  conducted  using  the  B-2  Computer-Assisted  Procedures  Trainer  (CAPT) 
resident  at  AL/CFHI,  and  made  use  of  the  following  five  distinct  hardware  elements: 

a.  Silicon  Graphics  4D-320  workstation  (IRIS3)  for  image  generation  and  cursor 
display. 

b.  Zenith  Z-248  PC  (linkage  computer)  for  sampling  of  subject  activity. 

c.  VT-220  terminal  hooked  to  the  SG  as  an  operator  console. 

d.  Top  center  MDU  at  the  right  seat  of  the  CAPT  for  display  of  imagery  to  the 
subject. 

e.  Track  handle  (i.e.,  cursor  controller)  at  the  right  seat  of  the  CAPT. 
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The  simulated  SAR  imagery  was  generated  via  the  software  in  the  SG  which  was  under 
the  control  of  the  operator  at  the  VT-220  console.  This  imagery  was  presented  to  the 
subject  on  a  1024x1024  pixel  non-interlaced  MDU  display.  This  MDU  had  two  times  the 
vertical  and  horizontal  resolution  of  the  MDU  currently  used  in  the  B-2  (512x512  pixels). 
Therefore,  for  the  experiment,  each  1024x1024  pixel  was  mapped  to  a  block  of  four 
pixels  to  create  the  appearance  of  a  512x512  display  and  to  emulate  the  aircraft  for 
cursor  positioning.  Thus,  the  term  “pixel"  referred  to  in  this  study  is  equivalent  to  a  pixel 
in  the  actual  aircraft.  Additional  hardware  and  software  were  incorporated  into  the  B-2 
CEF  baseline,  allowing  the  linkage  computer  to  accept  subject  inputs  from  the  CAPT 
track  handle  (cursor  movement  commands)  and  from  the  MDU  bezel  buttons  (image 
cycling  commands).  The  hardware  configuration  is  depicted  in  Figure  2.1. 


Figure  2.1 .  Cursor  Study  Hardware  Configuration. 


Stimuli 

The  stimuli  used  in  the  experiment  consisted  of  simulated  SAR  imagery  produced  by  a 
High-Resolution  Ground  Mapping  Radar  Simulator.  This  system  uses  an  improved 
Digital  Radar  Landmass  Simulator  (DRLMS)  database  and  applies  United  States 
Geological  Survey  (USGS)  high-altitude  photographic  data  as  source  imagery  combined 
with  a  variety  of  low-cost,  commercially-available  hardware  and  software  products  to 
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create  an  Image  synthesization  process  capable  of  generating  simulated  high-resolution 
radar  images.  A  total  of  40  unique  simulated  SAR  images  were  developed  for  use  as 
stimuli  in  the  experiment.  Of  the  five  patch  sizes  available  for  SAR  presentation  in  the 
B-2,  the  second  smallest  patch  size  was  used.  The  radar  fix  point  (RFP)  cards,  which 
the  subjects  examined  to  identify  the  aim  points,  were  generated  using  the  USGS  photo 
imagery.  Each  RFP  card  contained  a  verbal  description  of  the  target  and  a  small  circle 
drawn  around  the  target  in  the  photo.  In  the  center  of  that  circle,  a  single  pixel  was 
highlighted.  This  pixel  was  considered  to  be  the  golden  pixel  which  the  subjects  were 
instructed  to  designate.  The  RFP  cards  and  operational  procedures  were  highly 
representative  of  those  employed  in  the  aircraft.  An  example  of  an  RFP  card  is  shown 
in  Figure  2.2. 


Figure  2.2.  Radar  Fix  Point  Card . 
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Experimental  Variables 


The  independent  variable  for  the  study  was  the  type  of  cursor  used  for  the  task.  These 
cursors  have  been  described  in  detail  above.  The  three  cursor  types  were  evaluated  on 
two  criteria:  designation  speed  and  designation  accuracy,  which  are  defined  below. 

•  Designation  Speed:  The  elapsed  time  between  the  initial  appearance  of  the 
SAR  image  on  the  MDU  and  the  time  at  which  the  subject  designated  a  point 
on  the  image. 

•  Designation  Accuracy:  The  distance  (in  pixels)  between  the  golden  pixel  and 
the  actual  pixel  designated  (also  called  Designation  Error).  This  distance  was 
calculated  using  the  following  equation: 


Error d  =  (Error x  f  +  (Error 

where  Errors  is  the  overall  designation  error,  Errorx  is  the  error  in  pixels  on  the 
x  or  horizontal  axis,  and  Errory  is  the  error  on  the  y  or  vertical  axis. 

Procedure 

Prior  to  beginning  the  experimental  trials,  subjects  were  provided  with  written 
instructions  regarding  the  procedure  (see  Appendix  A,  Instructions  For  Radar  Cursor 
Study),  and  any  questions  they  had  were  discussed.  Ambient  room  lighting  was  set  at 
5  ft.  candles  to  approximate  representative  nominal  daytime  light  levels  in  the  cockpit. 
Each  subject  was  seated  in  the  Mission  Commander's  (right  hand)  seat  of  the  CAPT 
where  he  was  made  familiar  with  the  MDU  and  the  cursor  controller.  For  each 
experimental  trial,  the  subject  followed  the  following  procedures,  which  are  consistent 
with  those  employed  in  actual  mission  scenarios. 

•  Studied  the  appropriate  RFP  card  until  sufficiently  comfortable  with  the  aim 
point,  its  location,  and  associated  surroundings.  (No  time  restriction). 
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•  Depressed  the  aEXCa  button  located  in  the  upper  left  comer  of  the  center 
MDU.  This  action  initiated  the  trial  (a  computerized  timer  began  counting)  and 
the  SAR  image  appeared  on  the  MDU  with  the  cursor  positioned  in  the  center 
of  the  screen.  The  golden  pixel  was  offset  from  the  center  of  the  screen  by  a 
distance  that  varied  between  4  and  17  pixels  on  any  given  trial. 

•  Located  the  aim  point  that  was  circled  on  the  RFP  card  on  the  displayed  SAR 
image.  Using  the  "SLEW*  button  on  the  cursor  controller,  positioned  the  cursor 
on  the  aim  point,  precisely  over  the  golden  pixel  identified  on  the  RFP  card. 
Subjects  were  instructed  to  complete  the  designation  task  “as  accurately  and  as 
quickly  as  possible." 

•  Once  satisfied  with  the  cursor  placement,  the  subject  depressed  the  trigger 
on  the  cursor  controller  to  designate  the  aim  point.  The  action  ended  the  trial 
and  caused  the  MDU  to  go  blank  until  the  next  trial  was  initiated.  The  trial 
would  automatically  end  if  a  designation  was  not  made  within  60  seconds. 

Each  of  the  18  subjects  performed  a  total  of  150  trials,  of  which  30  were  practice  trials. 
As  depicted  in  Table  2.2,  the  first  15  trials  consisted  of  practice  trials  in  which  the 
subject  performed  five  trials  with  each  of  the  three  cursors.  The  subject  then  performed 
three  blocks  of  45  trials.  Within  each  block  the  subject  used  only  one  cursor.  To 
minimize  practice  effects  during  the  course  of  the  experimental  session,  the  order  in 
which  the  blocks  were  presented  was  counterbalanced  across  the  subjects.  In  addition, 
the  first  five  trials  of  each  block  were  considered  practice  trials  in  order  to  minimize 
practice  effects  as  each  new  cursor  was  introduced.  Thus,  each  subject  performed  40 
data  collection  trials  for  each  cursor  type,  or  a  total  of  120  data  collection  trials.  This 
resulted  in  a  total  of  2160  designation  performance  measures  collected. 
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Table  2.2.  Data  Collection  Approach. 


Practice  Test 


15 

Warm  up  trials  (3  cursor  Types) 

5 

40 

Block  1 

5 

40 

Block  2 

5 

40 

Block  3 

DataSet:  18  Subjects 

x  120  Trials 

2160  Data  Collection  Trials 


Subjective  Data  Collection 

Upon  completing  the  experimental  session,  subjects  participated  in  a  debriefing  session. 
During  this  interview  session,  a  questionnaire  was  used  and  responses  were  audio 
recorded.  Subjects  were  asked  questions  about  their  background  and  experience  that 
might  be  relevant  to  performance  data  collected  during  the  study.  In  addition,  they  were 
asked  a  number  of  questions  regarding  characteristics  of  the  cursors,  the  overall 
technique  or  strategy  they  used  for  cursor  positioning  and  designating  aim  points,  the 
cursor  controller  mechanization  (i.e.,  controlling  the  cursor  position),  and  the  realism  of 
the  part-task  simulation.  The  debriefing  questionnaire  is  contained  in  Appendix  B  and 
the  detailed  results  to  the  debriefing  sessions  are  contained  in  Appendix  C. 
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SECTION  III 
Results 

This  section  of  the  report  presents  the  results  of  the  analyses  performed  on  the 
performance  and  subjective  data  from  the  part-task  simulation. 

Performance  Results 

As  stated  earlier,  the  dependent  variables  of  interest  included  the  speed  and  accuracy 
(degree  of  error)  with  which  the  subjects  were  able  to  designate  the  golden  pixel  across 
the  three  cursor  types.  However,  additional  data  was  collected  that  tracked  the  position 
of  the  cursor  on  the  screen  during  the  course  of  each  trial.  The  goal  of  collecting  the 
cursor  position  and  movement  data  was  to  gain  an  insight  into  the  type  of  designation 
strategies  subjects  used  with  each  of  the  cursors.  In  addition,  it  provided  a  means  of 
evaluating  the  adequacy  of  the  cursor  controller  mechanization  by  indicating  the  number 
of  directional  reversals  in  cursor  movement  during  a  given  trial.  These  reversals 
indicated  the  overshooting  of  the  intended  designation  point. 

Descriptive  statistics  for  both  designation  speed  and  designation  error  are  presented 
below  in  Table  3.1 .  These  means  are  also  graphed  for  each  cursor  in  Figures 
3.1  a  and  b. 


Table  3.1.  Descriptive  Statistics  for  Designation  Speed  and  Error. 


Cursor 

Speed  (sec) 

Error  (pixels) 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

Cursor  1 

16.41 

8.90 

2.36 

2.28 

Cursor  2 

16.09 

8.35 

1.94 

2.06 

Cursor  3 

17.64 

10.35 

2.28 

2.52 
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Figure  3.1.  Mean  Designation  Speed  (a)  and  Designation  Error  (b)  for  Three  Cursors. 


As  depicted  in  Figure  3.1a,  designation  speed  was  very  similar  across  the  three 
cursors.  However,  designation  error  (Figure  3.1b)  appears  to  have  been  lower  (higher 
accuracy)  for  Cursor  2  than  for  the  other  cursors.  To  test  for  significant  differences 
among  the  speed  and  error  means,  two  one-way  Analyses  of  Variance  (ANOVA’s)  were 
conducted.  For  the  dependent  variable  Designation  Speed,  the  ANOVA  yielded  an 
F  =  1.23,  (df  =  2,34,  p.  =  .3042),  indicating  that  there  was  no  significant  difference 
between  means  for  designation  speed  across  the  three  cursors  at  alpha  =  .05.  For  the 
variable  Designation  Error,  however,  the  ANOVA  yielded  a  significant  difference: 

F  =  13.16,  (df  =  3,34,  p.  =  .0001).  A  Tukey  Studentized  Range  Test  was  then 
conducted  to  identify  the  significant  differences  among  means.  The  Tukey  test  yielded 
a  minimum  significant  difference  value  of  .212  (df  =  34,  alpha  =  .05,  MSE  =  2.694). 
Examining  the  means  for  Designation  Error  in  Table  3.1 ,  it  can  be  seen  that  the 
difference  between  Cursor  3  and  Cursor  2  (.34)  as  well  as  the  difference  between 
Cursor  1  and  Cursor  2  (.42),  exceed  the  minimum  significant  difference  of  .212, 
indicating  that  Cursor  2  elicited  significantly  lower  designation  error  (higher  accuracy) 
than  either  Cursor  1  or  Cursor  3,  which  were  not  statistically  different  from  each  other. 

Designation  Techniaue/Strategy 


A  number  of  subjects  reported  using  different  designation  strategies  for  the  different 
cursors.  This  is  discussed  in  more  detail  below  with  regard  to  the  subjective 
evaluations.  In  order  to  examine  the  difference  in  designation  strategy,  the  cursor 
position  data  was  examined.  The  mean  cursor  position,  in  terms  of  its  distance  from  the 
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golden  pixel,  was  calculated  across  all  trials  and  plotted  as  a  function  of  time  for  each  of 
the  three  cursors.  This  plot  is  depicted  in  Figure  3.2. 


Figure  3.2  shows  that  for  Cursor  1 ,  subjects  tended  to  slew  the  cursor  away  from  the 
golden  pixel  at  the  onset  of  the  trial  before  slewing  toward  the  golden  pixel  to  designate. 
For  Cursors  2  and  3,  however,  subjects  tended  to  slew  immediately  toward  the  golden 
pixel.  This  action  is  consistent  with  the  subjects’  comments  regarding  designation 
strategy  discussed  below. 

Mechanization 

Upon  analyzing  subjects'  comments  regarding  the  mechanization  of  the  cursor  controller 
(see  Subjective  Results),  it  became  clear  that  subjects  were  dissatisfied  with  their  ability 
to  accomplish  fine  positioning  of  the  cursor,  regardless  of  which  cursor  was  used. 
Subjects  commented  that  when  trying  to  position  the  cursor  on  the  golden  pixel,  they 
often  overshot  the  desired  pixel  repeatedly,  and  attributed  this  to  their  inability  to  easily 
move  the  cursor  a  distance  of  one  pixel  at  a  time.  Figure  3.3  below  illustrates  this  point, 
showing  a  random  trial  for  a  single  subject.  This  figure  shows  the  distance  from  the 
golden  pixel  in  the  x  and  y  dimensions  during  the  course  of  a  single  trial.  On  this  trial, 
the  subject,  repeatedly  maneuvered  the  cursor  to  the  appropriate  pixel  on  the  display  on 
both  the  x  and  the  y  dimensions.  However,  he  was  not  able  to  stop  the  movement  in 
time  to  keep  the  cursor  on  the  desired  pixel.  This  is  demonstrated  by  the  repeated 
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changes  in  both  the  horizontal  (x)  and  vertical  (y)  directions  and  transgressions  of  the 
O-error  point. 
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Figure  3.3.  Designation  Error  Over  Time  for  a  Single  Subject  and  a  Single  Trial. 

In  this  particular  example,  the  subject  was  using  Cursor  3.  However,  this  same  pattern 
of  difficulty  with  fine  positioning  could  be  seen  with  all  three  cursor  types. 

A  closer  examination  of  the  cursor  position  data  confirmed  that  subjects  were 
consistently  maneuvering  the  cursor  past  their  desired  designation  point.  The  number 
of  reversals  in  the  direction  of  cursor  movement  on  the  x  and  y  axes  was  counted  for 
each  trial  and  then  averaged  by  cursor  type  across  all  trials.  The  mean  number  of 
reversals  on  each  axis  is  listed  by  cursor  type  *.n  Table  3.2. 


Table  3.2.  Mean  Number  of  Reversals  in  Cursor  Movement  per  Trial. 


Cursor  Type 

Mean  Number  of  Reversals  Per  Trial 
x-axis  y-axis  Total 

1 

2.53 

2.30 

4.83 

2 

2.32 

2.14 

4.46 

3 

2.66 

2.44 

5.10 

Combined 

2.50 

2.29 

4.80 

The  data  in  Table  3.2  indicate  that  on  each  trial,  subjects  were  reversing  the  direction  of 
the  cursor  movement  an  average  of  four  to  five  times.  This  reversal  in  direction  of 
cursor  movement  reflects  the  fact  that  subjects  repeatedly  overshot  the  desired  pixel. 
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That  is,  they  would  move  in  the  direction  of  the  golden  pixel,  continue  past  it,  and  then 
reverse  direction  to  move  toward  it  again.  This  inability  to  control  fine  movements  of  the 
cursor  position  is  discussed  in  more  detail  below. 


The  following  provides  a  summary  of  the  results  of  the  performance  measures: 

•  Cursor  2  elicited  significantly  higher  designation  accuracy  (mean  error  =  1.94  pixels) 

than  the  current  cursor  (Cursor  1 ,  mean  error  =  2.36  pixels)  or  the  second  alternative 
cursor  (Cursor  3,  mean  error  =  2.28  pixels). 

•  Designation  speed  did  not  vary  significantly  across  cursors,  occurring  approximately 

17  seconds  into  a  trial  for  all  three  cursors. 

•  When  performing  the  radar  update  procedure  with  the  current  cursor  (Cursor  1), 

subjects  initially  moved  the  cursor  away  from  the  intended  designation  point,  before 
returning  to  the  aim  point  and  designating.  With  both  alternative  cursors,  subjects 
tended  to  move  directly  toward  the  intended  designation  point. 

•  When  moving  the  cursor  toward  the  intended  designation  point,  subjects  overshot  the 

intended  point  an  average  of  4.8  times  per  trial  across  all  three  cursors. 


This  section  presents  a  discussion  and  summary  of  the  responses  obtained  during  the 
debriefing  interview.  The  subjects'  responses  are  organized  into  the  following  four 
categories  based  on  the  questionnaire:  (1)  Evaluation  of  the  Cursor,  (2)  Description  of 
Designation  Technique,  (3)  Evaluation  of  the  Cursor  Mechanization,  and  (4)  Evaluation 
of  the  Part-Task  Simulation.  Appendix  B  consists  of  the  questionnaire  used  in  the 
debriefing  interview. 

A  summary  of  the  quantitative  and  qualitative  responses  to  each  questionnaire  item  is 
contained  in  Appendix  C.  For  each  of  the  rating  scales,  the  mean  and  standard 
deviation  are  presented.  In  addition,  summaries  of  the  comments  are  presented  for 
each  question. 
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During  the  debriefing  session,  subjects  were  asked  to  separately  rate  Cursors  1, 2, 
and  3  on  eight  distinct  dimensions  or  physical  characteristics.  A  five  point  anchored 
rating  scale  (1  =  very  difficult;  5  =  very  easy)  was  used.  The  following  figures 
graphically  display  the  results  of  the  rating  analysis  on  the  eight  dimensions. 


Very 

Difficult 


Average 


Very 

Easy 


Figure  3.4.  Visual  Localization.  Mean  ratings  for  three  cursor  types  on  the  dimension  of 
Visually  locating  or  seeing  the  cursor.”  Cl  =  2.78;  C2  =  3.89;  C3  =  4.06. 


Very 

Difficult 


Average 


Very 

Easy 


Figure  3.5.  Visualization  of  Cursor  Intersection.  Mean  ratings  for  three  cursor  types  on 
the  dimension  of  “Accurately  visualizing  the  center  of  the  cursor  intersection.” 
_  Cl  =2.28;  C2  =  4.17;  C3  =  4.11. 


Very 
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Average 


Very 

Easy 


Figure  3.6.  Cursor  Positioning.  Mean  ratings  for  three  cursor  types  on  the  dimension  of 
“Accurately  positioning  the  center  of  the  cursor  over  the  desired  designation  point.” 

Cl  =2.44;  C2  =  4.22;  C3  =  4.11. 
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Difficult  Easy 

Figure  3.7.  Cursor  Positioning  Speed.  Mean  rating  for  three  cursor  types  on  the 
dimension  of  “Quickly  positioning  the  center  of  the  cursor  over  the  desired  designation 
_  point."  Cl  =  3.1 1 ;  C2  =  3.72;  C3  =  3.72. _ 


Difficult  Easy 


Figure  3.8.  High-Clutter  Environment.  Mean  rating  for  three  cursor  types  on  the 
dimension  of  “Designating  aim  points  in  a  high-clutter  environment.” 
_ Cl  =  1 .83;  C2  =  3.44;  C3  =  3.39. _ 


Difficult  Easy 

Figure  3.9.  Uncluttered  Environment.  Mean  rating  for  three  cursor  types  on  the 
dimension  of  “Designating  aim  points  in  an  uncluttered  environment.” 
_ Cl  =  3.61;  C2  =  4.61;  C3  =  4.44. _ 
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Ineffective  Effective 


Figure  3.10.  Cursor  Effectiveness.  Mean  rating  for  three  cursor  types  on  the  dimension 
of  “Overall  effectiveness  of  each  cursor  type  for  designating  aim  points.” 
_ Cl  =  2.17;  C2  «  4.56;  C3  =  4.50. _ 


Figure  3.1 1 .  Cursor  Performance.  Number  of  respondents  choosing  each  cursor  on  the 
dimension  of  “Cursor  which  provided  best  overall  performance.” 
_ Cl  =  1;  C2  =  10;  C3  =  7. _ 


As  depicted  in  Figures  3.4  through  3.1 1,  subjects  consistently  rated  Cursors  2  and  3 
more  favorably  than  Cursor  1  in  terms  of  level  of  difficulty  and  overall  effectiveness  for 
all  eight  dimensions.  When  asked  to  rate  the  three  cursor  types  on  the  dimension  of 
“Visually  locating  or  seeing  the  cursor,”  subjects  found  Cursors  2  and  3  to  be  easier  to 
locate  due  to  the  longer  and  narrower  bars  and  the  inclusion  of  terminators  or  end 
points.  In  contrast,  they  found  that  Cursor  1  often  blended  into  the  background  when 
positioned  in  areas  containing  bright  returns,  thereby  obscuring  the  area  of  interest. 
Mean  ratings  for  “Accurately  visualizing  and  accurately  positioning  the  center  of  the 
cursor  over  the  desired  designation  points"  showed  Cursors  2  and  3  to  be  widely 
preferred  over  Cursor  1 .  Subjects  stated  that  Cursor  1  was  more  difficult  to  accurately 
visualize  and  accurately  position  due  to  the  large  center  opening  and  the  width  of  the 
cursor  bars.  When  asked  to  rate  the  three  cursor  types  on  the  dimension  of  “Quickly 
positioning  the  center  of  the  cursor  over  the  desired  designation  point,”  subjects 
expressed  difficulty  evaluating  all  cursors  on  speed  due  to  the  problems  they 
consistently  experienced  with  the  mechanization  (i.e.,  controlling  the  controller  position). 
They  did  state  however,  that  Cursor  1  required  an  extra  step  of  moving  away  from  the 
golden  pixel  in  order  to  see  the  area  of  interest.  Subjective  comments  demonstrated 
that  Cursors  2  and  3  were  preferred  when  designating  aim  points  in  a  high-clutter 
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environment.  Subjects  attributed  this  to  the  fact  that  Cursor  1  obscured  areas  of 
interest,  particularly  when  the  SAR  imagery  contained  a  number  of  bright  returns.  In 
contrast,  they  found  that  the  length  of  the  bars  and  the  end  points  on  Cursors  2  and  3 
allowed  them  to  line  up  past  bright  returns. 

As  depicted  in  Figures  3.10  and  3.1 1,  Cursors  2  and  3  were  found  to  be  superior  to 
Cursor  1  in  terms  of  overall  effectiveness  and  performance.  Subjects  found  Cursor  1 
to  be  “somewhat  ineffective”  due  to  the  width  of  the  bars,  the  length  of  the  bars,  large 
center  opening,  and  overall  size.  Those  subjects  who  favored  Cursor  2  stated  that  they 
preferred  the  overall  size,  narrow  bars,  and  perpendicular  end  points.  Subjects  who 
found  Cursor  3  to  be  superior  cited  the  versatility  of  the  design;  wide  end  points  to 
acquire  the  cursor  and  for  performing  gross  positioning,  and  narrow,  internal  bars  for 
fine  positioning  and  final  designation. 

Finally,  subjects  were  asked  to  suggest  improvements  to  the  cursor  they  found  to  be  the 
best  overall  performer.  Although  the  majority  of  the  subjects  did  not  feel  that 
improvements  to  Cursors  2  and  3  were  required,  a  few  subjects  who  selected  Cursor  2 
recommended  that  the  overall  size  could  be  further  increased  (i.e.,  the  bars 
lengthened);  those  subjects  who  selected  Cursor  3  recommended  the  inclusion  of  a 
blinking  or  flashing  capability  to  maximize  the  contrast  between  the  cursor  and  the 
background. 

Description  of  Designation  Technioue/Strategv 

During  the  debriefing  session,  subjects  were  asked  to  describe  the  overall  technique  or 
strategy  they  used  for  cursor  positioning  and  designating  aim  points.  Similarities  and 
differences  for  each  of  the  cursors  were  recorded.  As  described  earlier  in  this  report,  a 
number  of  subjects  reported  using  a  different  strategy  for  Cursor  1  than  Cursor  2  and 
Cursor  3.  For  Cursor  1 ,  subjects  tended  to  slew  the  cursor  away  from  the  golden  pixel 
at  the  onset  of  the  trial  in  order  to  uncover  important  information  in  the  area  of  interest. 

In  contrast,  subjects  were  able  to  slew  immediately  toward  the  golden  pixel  when  using 
Cursors  2  and  3.  These  comments  are  consistent  with  the  performance  data  depicted 
in  Figure  3.2  (Mean  Designation  Error  over  Time). 

Subjects  stated  that  they  were  able  to  accomplish  fine  positioning  more  easily  with 
Cursors  2  and  3.  They  attributed  this  to  another  strategy  they  reported  to  be  effective 
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which  involved  using  the  longer  and  thinner  bars  of  Cursors  2  and  3  to  line  up  on  the 
golden  pixel  one  axis  at  a  time.  Because  of  the  larger  opening  and  shorter,  wider  bars 
on  Cursor  1,  subjects  found  they  were  unable  to  use  this  same  strategy  with  the  current 
cursor. 

Evaluation  of  the.Cujsor  Mechanization 

During  the  debriefing  session,  subjects  were  asked  to  rate  the  difficulty  of  controlling  the 
cursor  position  in  terms  of  designation  speed  and  accuracy.  A  five  point  anchored 
rating  scale  (1  =  very  difficult;  5  =  very  easy)  was  used.  Results  yielded  a  mean  rating 
on  the  dimension  of  mechanization  of  1 .72,  indicating  that  subjects  found  it  difficult  to 
control  the  cursor  position.  Regardless  of  cursor  type,  subjects  consistently  expressed 
dissatisfaction  with  their  ability  to  control  the  cursor,  particularly  in  the  area  of  fine 
positioning.  As  stated  previously,  subjects  commented  that  when  trying  to  position  the 
cursor  on  the  golden  pixel,  they  often  overshot  the  desired  pixel  repeatedly,  and 
attributed  this  to  their  inability  to  easily  move  the  cursor  a  distance  of  one  pixel  at  a  time. 
As  depicted  in  Figure  3.3  (Designation  Error  over  Time  for  a  Single  Subject  and  a  Single 
Trial)  and  Table  3.2  (Mean  Number  of  Reversals  in  Cursor  Movement  per  Trial),  the 
performance  data  supported  these  subjective  comments  reported  during  the  debriefing. 
Suggestions  for  improvement  included  (1)  readjusting  the  slew  rate  to  decrease 
overshooting  and  thereby  improving  fine  positioning  accuracy,  and  (2)  investigating  the 
use  of  a  dual-rate  (i.e.,  fast  and  slow)  mechanization  for  gross  and  fine  positioning. 

A  number  of  subjects  objected  to  the  cursor  controller  itself.  They  stated  that  too  much 
pressure  or  force  was  required  to  move  the  cursor.  In  addition,  subjects  had  difficulty 
locating  the  "sensitive  area"  on  the  controller.  Suggestions  for  improvement  in  this  area 
included:  (1)  decreasing  the  deadband,  (2)  greater  movement  of  the  controller  in  order 
to  provide  feedback,  (3)  proportional  movement  for  force  applied,  and  (4)  possibly  using 
a  thumbwheel  or  trackball  to  provide  greater  back  and  forth  movement. 

In  conclusion,  many  subjects  found  the  current  mechanization  to  be  the  source  of 
greatest  concern  when  accomplishing  the  radar  update  procedure.  They  stated  that 
although  they  knew  the  location  of  the  golden  pixel,  many  times  they  were  unable  to 
position  the  cursor  over  the  desired  designation  point  due  to  the  mechanization.  They 
objected  to  having  to  employ  a  bumping  strategy  (i.e.,  tapping  the  controller  in  an 


21 


attempt  to  move  the  cursor  one  pixel  at  a  time).  All  subjects  felt  that  if  the 
mechanization  was  improved,  radar  update  accuracy  would  also  greatly  improve. 


Evaluation  of  the  Part-Task  Simulation 

In  the  final  section  of  the  debriefing  questionnaire,  subjects  were  asked  to  evaluate  the 
realism  of  the  part-task  simulation.  Specifically,  subjective  comments  were  recorded  in 
the  areas  of  simulated  SAR  imagery  quality,  the  procedure  used  to  accomplish  the  radar 
updates,  mechanization  of  the  cursor,  and  the  quality  of  the  RFP  cards.  As  depicted  in 
Table  3.3,  subjective  evaluations  of  the  simulation  indicated  a  high  degree  of  fidelity  on 
the  above  mentioned  parameters. 


Table  3.3.  B-2  Cursor  Study  Debrief  Results. 


■■■■■I 

E555S3 

MAXIMUM 

RATING 

POSSIBLE 

NEAN 

STAMMRD 

DEVIATION 

Rate  realism  of  simulated  SAR  imagery 
presented  during  demonstration. 

17 

3.00 

2.67 

.61 

Aim  point  types  typical  of  those  you 
experience  in  your  operational  units? 

17 

Yes/No 

100%  = 
Yes 

l 

■ 

1 

\ 

i 

i 

Rate  preset  CONTRAST  and 
BRIGHTNESS  levels  in  terms  of 
desirability. 

18 

3.00 

2.44 

.62 

Rate  effectiveness  of  demonstration  in 
simulating  the  radar  update  procedure 

16 

5.00 

4.44 

1.03 

Rate  effectiveness  of  demonstration  in 
simulating  the  mechanization  of  the 
cursor. 

8 

5.00 

m 

1.19 

Rate  realism  of  RFP  cards. 

15 

3.00 

mm 

.51 

In  terms  of  realism  of  the  simulated  SAR  imagery,  subjects  rated  the  imagery  as  being 
realistic  (mean  rating  =  2.67  out  of  maximum  rating  of  3).  When  asked  what  SAR 
effects/characteristics  would  you  change,  three  subjects  suggested  improvement  in  the 
area  of  shadowing.  The  aim  points  selected  for  the  study  were  found  to  be  typical  of 
those  seen  in  actual  missions  and  subjects  found  the  radar  procedure  to  be  very 
realistic  in  terms  of  what  is  performed  in  an  operational  environment  (mean  rating  =  4.44 
out  of  maximum  rating  of  5).  Of  particular  interest  was  the  mean  rating  on  the 
effectiveness  of  the  mechanization  of  the  cursor.  Although  only  eight  of  the  subjects  felt 
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they  had  sufficient  experience  to  rate  this  dimension,  those  eight  subjects  found  the 
mechanization  to  be  very  effective.  That  is.  they  found  the  study  to  have  effectively 
simulated  the  current  mechanization  of  the  B-2. 
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Discussion 


SECTION  IV 

Discussion  and  Recommendations 


Subjective  evaluations  of  the  current  part-task  simulation  effort  indicate  that  the  study 
approach  exhibited  a  high  degree  of  fidelity  with  operational  environment  (i.e.,  SAR 
imagery  quality,  procedure  used  to  accomplish  the  radar  updates,  mechanization  of 
cursor,  and  quality  of  the  RFP  cards.)  In  addition,  the  CEF  B-2  crewmembers  who 
participated  as  advisors  and  subjects  rated  the  hardware  and  software  implementations 
as  being  highly  representative  of  those  in  the  B-2  aircraft.  Thus,  it  is  assumed  that  the 
results  of  the  experimentation,  in  terms  of  relative  effectiveness  of  the  three  cursors 
evaluated,  can  be  generalized  to  the  operational  environment. 

Both  the  performance  and  subjective  results  clearly  indicate  that  the  radar  cursor 
currently  being  used  in  the  B-2  could  be  modified  to  elicit  more  accurate  designation 
and  a  higher  degree  of  user  acceptance.  Crewmember  complaints,  which  served  as  the 
impetus  for  conducting  this  study,  were  validated  during  the  course  of  the 
experimentation.  That  is,  subjects  in  the  study  demonstrated  difficulty  with  the  current 
cursor  in  terms  of:  (1)  accurately  placing  the  cursor  with  the  precision  of  one  pixel, 

(2)  seeing  relevant  target  information  that  was  obscured  by  the  cursor,  (3)  determining 
the  exact  center  of  the  cursor,  (4)  locating  the  cursor  in  a  high-clutter  environment,  and 
(5)  controlling  fine  positioning  of  the  cursor. 

Perhaps  the  most  vital  aspect  of  performance  in  an  aim  point  designation  task  is  the 
accuracy  with  which  the  operator  can  designate  the  desired  pixel.  Error  in  this  process 
is  manifested  in  terms  of  imprecise  radar  updates  and  reduced  accuracy  of  both  the 
navigation  and  bombing  solutions.  Some  degree  of  the  designation  error  observed  with 
the  current  cursor  was  thought  to  be  due  to  the  wide  and  short  cursor  bars  (3  pixels 
wide  by  ten  pixels  long)  and  the  rather  large  cursor  opening  (7x7  pixels).  Subjective 
comments  verified  that  subjects  felt  both  these  areas  were  a  problem  in  terms  of 
accuracy.  Subjects  rated  the  current  cursor  (Cursor  1)  “somewhat  ineffective”  due  to 
the  width  of  the  bars,  the  length  of  the  bars,  large  center  opening,  and  overall  size. 

They  preferred  Cursors  2  and  3  because  of  the  overall  larger  size,  narrower  bars, 
smaller  center  opening,  and  inclusion  of  end  points.  Performance  results  did  in  fact 
show  that  Cursors  2  and  3,  with  thinner  and  longer  bars  and  smaller  openings  led  to 
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lower  mean  designation  error  than  Cursor  1 .  As  stated  previously,  Cursor  2  elicited 
significantly  higher  designation  accuracy  (mean  error  =  1 .94  pixels)  than  the  current 
cursor  (Cursor  1 ,  mean  error  =  2.36  pixels)  or  the  second  alternative  cursor  (Cursor  3, 
mean  error  =  2.28).  However,  the  difference  in  designation  error  was  only  statistically 
significant  between  Cursor  2  and  Cursors  1  and  3.  That  is,  Cursor  2  elicited  significantly 
lower  error  than  both  Cursor  1  and  3.  The  improved  performance  of  Cursor  3  over 
Cursor  1  was  not  statistically  significant.  This  fact  indicates  that  the  above  mentioned 
modifications  (i.e.,  overall  larger  size,  narrow  bars,  smaller  center  opening,  and  end 
points)  may  not  have  been  the  only  factors  affecting  designation  accuracy. 

Extension  of  the  length  of  the  cursor  bars  (Cursors  2  and  3)  elicited  a  higher  degree  of 
user  acceptance  in  terms  of  the  subjects'  ability  to  locate  the  cursor  on  the  screen. 
Subjective  comments  indicate  that  the  longer  (although  narrower)  bars  of  Cursors  2  and 
3  made  the  cursor  easier  to  locate  when  the  SAR  display  first  appeared.  This  effect 
was  described  by  subjects  to  be  exaggerated  when  the  cursor  appeared  in  an  area  of 
high  returns.  The  resulting  effect  on  mission  performance  of  an  inability  to  initially  locate 
the  cursor  would  seem  to  be  a  delay  in  designation  time.  However,  a  statistically 
significant  difference  in  designation  time  across  cursors  was  not  observed.  Although 
subjects  expressed  that,  at  times,  they  had  difficulty  locating  Cursor  1  on  the  display 
screen,  performance  data  indicated  that  this  did  not  affect  the  overall  time  it  took  them 
to  designate. 

Cursors  2  and  3  were  widely  preferred  over  Cursor  1  in  terms  of  overall  performance. 

Of  the  subjects  who  preferred  Cursor  3  over  the  others,  many  indicated  that  they 
preferred  the  longer  terminator  bars  because  they  facilitated  the  act  of  locating  the 
cursor  as  the  display  first  appeared.  As  discussed  above,  this  had  no  significant  impact 
on  designation  time.  However,  those  subjects  who  felt  that  Cursor  2  offered  the  best 
performance  attributed  this  to  the  fact  that  the  perpendicular  terminators  on  Cursor  2 
aided  not  only  in  their  locating  the  cursor,  but  also  in  their  accurate  alignment  of  the 
cursor  and  subsequent  designation.  That  is,  the  perpendicular  terminators  may  provide 
better  vernier  alignment  cues,  enabling  subjects  to  more  easily  align  the  cursor  over  the 
intended  designation  point.  These  comments,  coupled  with  the  performance  results, 
suggest  that  the  increased  designation  accuracy  elicited  by  Cursor  2  could  be  attributed 
to  both  the  smaller  cursor  opening  and  the  perpendicular  terminators. 
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Problems  associated  with  obscuration  of  the  imagery  and  subjects'  inability  to  easily 
locate  the  cursor  were  evaluated  for  the  current  cursor  and  for  both  alternative  cursors. 
Although  not  all  subjects  found  obscuration  to  be  a  problem  with  Cursor  1 ,  many 
subjects,  when  explaining  their  strategy  of  initially  moving  the  cursor  away  from  the 
designation  point,  commented  that  they  found  its  bars  to  be  too  wide  (three  pixels). 

With  Cursors  2  and  3,  subjects  did  not  find  it  necessary  to  move  the  cursor  away  from 
the  target  before  designating  (see  Figure  3.2).  Discussions  with  the  subjects  revealed 
that  this  was  due  to  the  fact  that  Cursors  2  and  3,  with  bars  of  only  one  pixel  in  width, 
did  not  tend  to  obscure  vital  visual  information  in  the  imagery. 

Regardless  of  the  physical  characteristics  of  the  current  cursor  and  the  two  alternative 
cursors  tested,  subjects  consistently  expressed  dissatisfaction  with  their  ability  to  control 
the  cursor,  particularly  in  the  area  of  fine  positioning.  Subjects  commented  that  when 
trying  to  position  the  cursor  on  the  golden  pixel,  they  often  overshot  the  desired  pixel 
repeatedly,  and  attributed  this  to  their  inability  to  easily  move  the  cursor  a  distance  of 
one  pixel  at  a  time.  Suggestions  for  improvement  included  both  hardware  and  software 
changes  (i.e.,  decreasing  deadband,  greater  movement  of  the  controller  in  order  to 
provide  increased  feedback,  readjusting  slew  rate  to  improve  fine  positioning).  In 
general,  subjects  found  the  current  mechanization  to  be  the  source  of  greatest  concern 
when  accomplishing  the  radar  update  procedure  and  felt  that  if  the  mechanization  was 
improved,  radar  update  accuracy  would  also  greatly  improve. 

Recommendations 

Based  on  the  performance  results  and  subjective  comments  resulting  from  this  test,  it  is 
suggested  that  Cursor  2  would  yield  an  improvement  over  the  current  cursor  in  terms  of 
a  higher  degree  of  designation  accuracy  as  well  as  a  higher  degree  of  user  acceptance. 
However,  the  increase  in  designation  accuracy  (from  a  mean  error  of  2.36  pixels  with 
the  current  cursor  to  an  mean  error  of  1 .94  pixels  with  Cursor  2)  does  not  meet  the 
specified  requirement  of  a  mean  designation  error  of  one  pixel  or  less. 

Based  on  the  overwhelming  number  of  negative  responses  from  subjects  and  B-2 
crewmembers  regarding  the  cursor  controller  mechanization,  it  is  apparent  that  the 
inability  to  finely  position  the  cursor  may  be  equally,  if  not  more,  responsible  for  the  error 
in  aim  point  designation  than  the  physical  characteristics  of  the  cursor.  As  a  result  of 
these  subjective  evaluations,  it  is  recommended  that  the  focus  of  any  future 
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investigation  of  designation  accuracy  be  directed  toward  optimizing  the  control 
mechanization  of  the  cursor.  An  increased  ability  to  make  fine  cursor  movements, 
coupled  with  the  improved  physical  design  of  the  cursor,  may  lead  to  accuracy 
achieving  the  criterion  of  a  single  pixel. 
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GLOSSARY  OF  TERMS 


Cursor.  The  symbol  displayed  on  the  multipurpose  display  unit  (MDU)  that  moves  as 
the  radar  operator  moves  the  slew  button  on  the  cursor  controller.  In  this  experiment, 
the  cursor  was  used  to  designate  aim  points. 

Dependent  Variable.  Some  well-defined  aspect  of  behavior  (a  response)  that  is 
measured  in  a  study.  The  assumed  value  of  the  dependent  variable  is  hypothesized  to 
be  dependent  upon  the  value  assumed  by  the  independent  variable  and  thus  is 
expected  to  systematically  change  in  response  to  relative  changes  in  the  independent 
variable.  The  dependent  variable  reflects  any  effects  associated  with  the  manipulation 
of  the  independent  variable.  In  the  current  study,  the  dependent  variables  were 
designation  speed  and  designation  error. 

Experimental  Design.  A  specific  plan  used  to  systematically  vary  independent  variables 
and  note  consequent  changes  in  dependent  variables. 

Golden  Pixel.  The  specific  pixel  on  the  RFP  card  and  corresponding  SAR  image  that 
subjects  were  instructed  to  designate. 

Independent  Variable.  An  aspect  of  the  testing  environment  that  is  empirically 
investigated  for  the  purpose  of  determining  whether  or  not  it  influences  the  experimental 
outcome.  Test  variables  or  stimuli  deliberately  varied  or  controlled  within  the 
experiment  to  determine  their  effects  on  the  dependent  variables.  In  the  current  study, 
the  independent  variable  was  cursor  type. 

Trial.  A  discrete  data  collection  event  consisting  of  a  single  set  of  independent  and 
dependent  variables. 
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APPENDIX  A 


Instructions  for  Radar  Cursor  Study 


29 


INTRODUCTION  TO  B-2  RADAR  CURSOR  DEMONSTRATION 


You  are  invited  to  participate  in  a  simulator-based  demonstration  that  involves 
performing  radar  updates  using  simulated  synthetic  aperture  radar  (SAR)  imagery.  As 
you  know,  the  purpose  of  performing  these  updates  is  to  ensure  the  accuracy  of  the 
navigation  and  bombing  solutions.  The  objectives  of  this  study  are: 

•  To  demonstrate  the  functionality  of  the  three  radar  cursors  being  tested. 

•  To  evaluate  the  realism  of  the  computer  generated  SAR  imagery. 

•  To  assess  the  adequacy  of  the  man-machine  interface  used  to  support  this 
task. 

The  demonstration  will  be  conducted  using  the  B-2  Computer  Assisted  Procedures 
Trainer  (CAPT)  resident  at  Armstrong  Laboratory  (AL/CFHI).  This  simulator,  typically 
used  for  stand-alone  B-2  procedures  training,  has  been  modified  to  allow  AL  scientists 
to  evaluate  B-2  controls  and  displays  in  an  interactive  environment.  Do  not  be 
concerned  if  you  are  not  familiar  with  the  B-2  CAPT,  as  an  in-depth  knowledge  will  not 
be  required.  You  will  be  well  trained  on  all  procedures  necessary  to  participate  in  the 
study. 

In  order  to  allow  adequate  time  for  training,  participation  in  the  demonstration,  and 
debriefing,  we  are  requesting  one  half  day  of  your  time  (either  0800  to  1200  or  1300  to 
1700)  to  be  spent  in  our  facility.  A  schedule  for  the  half  day  session  will  be  as  follows: 


Time _ Activity 


AM 

PM 

0800-0830 

1300-1330 

Orientation  Briefing 

0830-0845 

1330-1345 

Practice  Session 

0845-0930 

1345-1430 

Demonstration  Block  1 

0930-0945 

1430-1445 

Break 

0945-1115 

1445-1615 

Demonstration  Blocks  2  &  3 

1115-1200 

1615-1700 

Debriefing 
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The  following  is  a  brief  description  of  the  demonstration,  the  specific  procedures  that 
you  will  be  following,  and  the  materials  or  tools  you  will  be  using. 

DEMONSTRATION  DESCRIPTION 

During  the  demonstration,  you  will  be  presented  with  a  simulated  SAR  imagery  on  the 
mission  commander's  center  multipurpose  display  unit  (MDU).  All  imagery  will  be 
presented  NORTH  UP  and  contain  representative  aim  points  typical  of  those  used  in  an 
operational  bomber  unit  (e.g.,  tanks,  bridges,  buildings,  towers,  etc.).  Assume  that  the 
offensive  avionics  are  operating  normally  and  that  a  nominal  buffer  value  exists.  For 
each  image  presented,  you  will  be  asked  to  identify  and  designate  an  aim  point  as 
depicted  on  the  Radar  Fix  Point  (RFP)  Graphic  Card  provided.  Once  the  aim  point  is 
designated,  the  trial  will  end.  It  is  important  that  you  designate  aim  points  for  each  of 
the  trials  as  accurately  as  possible.  In  addition,  designation  should  occur  within  a  time 
frame  that  is  consistent  with  an  operational  setting.  Please  note  that  the  CONTRAST 
and  BRIGHTNESS  controls  have  been  preset  and  that  you  will  be  unable  to  adjust  their 
levels. 


DEMONSTRATION  PROCEDURES  AND  TOOLS 

The  following  describes  the  sequence  of  events  for  each  trial,  the  displays  you  will  see, 
the  controls  you  will  use  to  make  your  inputs,  and  the  materials  that  will  be  provided. 

1.  Radar  Fix  Point  (RFP)  Graphic  Cards.  RFP  cards  have  been  developed  to  support 
each  simulated  SAR  image  for  each  trial.  Contained  on  each  RFP  card  is  a 
photographic  image  of  the  desired  aim  point  and  the  surrounding  area.  The  orientation 
of  the  RFP  card  will  always  be  presented  NORTH  UP.  The  aim  point  contained  on  each 
photograph  will  be  circled  to  indicate  the  target  you  are  to  designate  for  that  particular 
trial.  Your  job  will  be  to  study  the  RFP  card  for  a  short  length  of  time  prior  to  the 
beginning  of  a  trial.  The  time  at  which  a  trial  begins  is  under  your  control,  thereby 
allowing  you  to  study  the  card  as  long  as  you  would  like.  RFP  cards  are  labeled  in  the 
bottom  left  corner,  indicating  its  associated  block  and  trial  number.  For  example,  Block 
1,  image  (i.e.,  trial)  30  would  be  designated  as  “B1-30." 


31 


2.  Trial  Procedure.  The  procedure  you  will  use  for  designating  aim  points  is  as  follows: 


a.  Locate  the  correct  RFP  card  for  the  upcoming  trial.  Study  the  RFP  card  until  you 
are  sufficiently  comfortable  with  the  aim  point,  its  location,  and  associated 
surroundings. 

b.  Depress  the  "EXC"  button  from  coherent  map  display.  This  button  is  located  in  the 
upper  left  comer  of  the  center  MDU.  This  action  initiates  the  trial  and  will  cause  the 
simulated  SAR  image  to  be  displayed. 

c.  Once  the  SAR  image  is  displayed,  note  that  the  image  you  are  seeing  on  the  MDU 
is  the  same  image  that  is  depicted  on  the  RFP  card.  The  crosshair  or  cursor  is 
displayed  in  the  center  of  the  display. 

d.  Locate  the  aim  point  that  was  circled  on  the  RFP  card  on  the  displayed  SAR  image. 
Using  the  SLEW  button  on  the  Cursor  Controller,  position  the  cursor  on  the  aim 
point,  precisely  over  the  “golden  pixel"  identified  on  the  RFP  card.  Remember 
accuracy  is  important,  as  these  updates  become  part  of  the  navigation  and  bombing 
solution  on  a  real  mission. 

e.  When  you  are  satisfied  with  your  aiming,  depress  the  trigger  on  the  Cursor 
Controller  to  designate  the  aim  point.  The  action  ends  the  trial  and  causes  the  MDU 
to  be  blanked  until  you  initiate  the  next  trial. 

*  Please  complete  the  location  and  designation  procedure  as  accurately  and  as 
quickly  as  possible. 

This  procedure  will  be  repeated  for  a  total  of  135  images  or  trials.  These  trials  will  be 
organized  into  three  groups  or  blocks  of  45  images  each.  Each  block  will  present  a 
different  radar  cursor  for  you  to  use.  The  three  cursors  will  vary  in  size  and  style.  Block 
1  will  contain  45  images  or  trials  using  one  type  of  cursor;  Block  2  will  contain  45  images 
using  another  type  of  cursor;  and  finally  Block  3  will  contain  45  images  using  still  a 
different  cursor. 

3.  Practice  Session.  As  stated  previously,  you  will  participate  in  a  practice  session  prior 
to  the  beginning  of  the  demonstration.  Using  the  procedure  described  above,  you  will 
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be  given  time  to  practice  designating  aim  points  using  the  three  radar  cursors  being 
tested.  This  session  will  last  approximately  fifteen  minutes  and  may  be  repeated  until 
both  you  and  the  experimenter  feel  that  you  are  comfortable  with  the  procedure. 

DEBRIEFING  INTERVIEW 

At  the  completion  of  the  demonstration,  you  will  be  asked  to  participate  in  a  debriefing 
session.  You  will  be  asked  questions  about  your  background  and  experience  that  may 
be  helpful  to  us  in  interpreting  the  performance  data  we  collect  during  the 
demonstration.  We  will  also  be  asking  you  questions  about  the  functionality  of  the  three 
radar  cursors  that  were  used.  Finally,  we  will  be  asking  you  to  evaluate  the 
demonstration  to  include  the  realism  of  the  imagery,  the  procedure  used  to  accomplish 
the  updates,  and  the  operation  of  the  Cursor  Controller. 
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APPENDIX  B 


B-2  Radar  Curaor  Study  Quaatlonnalra 
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B-2  RADAR  CURSOR  STUDY  QUESTIONNAIRE 


SECTION  1:  BIOGRAPHICAL  DATA 

The  following  section  contains  questions  regarding  your  background  and  previous 
experience  that  might  be  relevant  to  the  B-2  radar  cursor  demonstration.  The 
information  that  you  provide  in  this  section  may  be  helpful  to  us  in  interpreting  the 
data  we  collect  during  the  demonstration. 

A.  Personal  Data _ _ 

Name: _ Rank: _ 

Organization:  Phone: 

B.  Experience 

1 .  List  aircraft  in  which  you  are  currently  qualified,  your  crew  position,  and 
number  of  hours. 

total 

_ AIRCRAFT _ POSITION _ HOURS 

1.  — y~ 

2. 

2.  List  additional  aircraft  in  which  you  have  been  qualified  in  the  past,  your  crew 

position,  and  number  of  hours. _ 


TOTAL 

AIRCRAFT _ _  POSITION _ HOURS 


3.  Which  of  the  following  sensors  have  you  operated  and  how  many  hours  do  you 
have  with  each  type? 


_ SENSOR _ AIRCRAFT _ HOURS 

EVS/FLIR 

SAR _ 

Other  _ 

4.  Describe  your  experience  operating  a  crosshair  using  a  cursor  controller 

(ex.  trackball,  trackhandle)  to  include  flight  environment  (high/low  altitude)  and  hours. 
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SECTION  2:  EVALUATION  OF  THE  CURSOR 


The  following  section  contains  questions  regarding  the  three  cursors  tested,  the 
technique  you  used  to  designate  aim  points,  and  the  mechanization  of  the  cursors. 


CURSOR  1 


CURSOR 2 


I  ■  ..I 


CURSOR  3 


L——J 


A.  Cursor  Type 

The  following  questions  pertain  to  the  characteristics  of  the  cursors  themselves.  For 
each  of  the  following  categories,  please  rate  each  of  the  three  cursors  on  a  scale  of 
1  to  5. 

How  would  you  rate  the  level  of  difficulty  for  performing  the  following  activities. 

1.  In  general,  visually  locating  or  seeing  the  cursor. 


□  l.  Very  difficult 

□  2.  Somewhat  difficult 

□  3.  Average 

□  4.  Somewhat  easy 

□  5.  Very  easy 

□  l.  Very  difficult 

□  2.  Somewhat  difficult 

□  3.  Average 

□  4.  Somewhat  easy 

□  5.  Very  easy 

□  l.  Very  difficult 

□  2.  Somewhat  difficult 

□  3.  Average 

□  4.  Somewhat  easy 

□  5.  Very  easy 

Comments: 

1 

KLiiXi] 

2 

3 

I 


2.  Accurately  visualizing  the  center  of  the  cursor  intersection. 


□  l.  Very  difficult 

□  2.  Somewhat  difficult 

□  3.  Average 

□  4.  Somewhat  easy 

□  5.  Very  easy 

□  l.  Very  difficult 

□  2.  Somewhat  difficult 

□  3.  Average 

□  4.  Somewhat  easy 

□  5.  Very  easy 

□  l.  Very  difficult 

□2.  Somewhat  difficult 

□  3.  Average 

□  4.  Somewhat  easy 

□  5.  Very  easy 

Comments: 

3.  Accurately  positioning  the  center  of  the  cursor  over  the  desired  designation 
point. 

Cursor  1 

Cursor  2 

Cursor  3 

□  l.  Very  difficult 

□  l.  Very  difficult 

□  l.  Very  difficult 

□  2.  Somewhat  difficult 

□  2.  Somewhat  difficult 

□  2.  Somewhat  difficult 

□  3.  Average 

□  3.  Average 

□  3.  Average 

□  4.  Somewhat  easy 

□  4.  Somewhat  easy 

□  4.  Somewhat  easy 

□  5.  Very  easy 

□  5.  Very  easy 

□  5.  Very  easy 

Comments: 

1 

2 

I  » 


r  "U  i 


4.  Quickly  positioning  the  center  of  the  cursor  over  the  desired  designation  point. 


□  l.  Very  difficult 

□  2.  Somewhat  difficult 

□  3.  Average 

□  4.  Somewhat  easy 

□  5.  Very  easy 


Comments: _ 


□  l.  Very  difficult 

□  2.  Somewhat  difficult 

□  3.  Average 

□  4.  Somewhat  easy 

□  5.  Very  easy 


□  l.  Very  difficult 

□  2.  Somewhat  difficult 

□  3.  Average 

□  4.  Somewhat  easy 

□  5.  Veryeas 


5.  Designating  aim  points  in  a  high-clutter  environment. 


CuiSQLl 

□  l.  Very  difficult 

□  2.  Somewhat  difficult 

□  3.  Average 

□  4.  Somewhat  easy 

□  5.  Very  easy 


Comments: _ 


Cursor  2 

□  l.  Very  difficult 

□  2.  Somewhat  difficult 

□  3.  Average 

□  4.  Somewhat  easy 

□  5.  Veryeas 


C.ycaor-2 

□  l.  Very  difficult 

□  2.  Somewhat  difficult 

□  3.  Average 

□  4.  Somewhat  easy 

□  5.  Veryeas 
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6.  Designating  aim  points  in  an  uncluttered  environment. 


Cursor  1 

□  1 .  Very  difficult 

□  2.  Somewhat  difficult 

□  3.  Average 

□  4.  Somewhat  easy 

□  5.  Very  easy 


omments: _ 


□  1 .  Very  difficult 

□  l.  Very  difficult 

□  2.  Somewhat  difficult 

□  2.  Somewhat  difficult 

□  3.  Average 

□3.  Average 

□  4.  Somewhat  easy 

□  4.  Somewhat  easy 

□  5.  Very  easy 

□  5.  Very  easy 

7.  Please  rate  the  overall  effectiveness  of  each  cursor  type  for  designating  aim 
points. 


Cursor, .1  Cursors  Cursor.  3 

□  l.  Very  ineffective  □  1.  Very  ineffective  □  1.  Very  ineffective 

□  2.  Somewhat  ineffective  □  2.  Somewhat  ineffective  □  2.  Somewhat  ineffective 

□  3.  Average  D3.  Average  03.  Average 

□  4.  Somewhat  effective  □  4.  Somewhat  effective  □  4.  Somewhat  effective 

□  5.  Very  effective  D5.  Very  effective  D5.  Very  effective 


□  4.  Somewhat  effective 

□  5.  Very  effective 


Comments: _ 


Cursor.  3 

□  l.  Very  ineffective 


□  4.  Somewhat  effective 

□  5.  Very  effective 
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C.  Mechanization 

The  following  questions  address  the  mechanization  characteristics  of  the  cursor 
(regardless  of  cursor  type). 

1 .  In  terms  of  designation  speed  and  accuracy,  how  would  you  rate  the  difficulty  of 
controlling  the  cursor  position? 

□  l.  Very  difficult 

□  2.  Somewhat  difficult 

□  3.  Average 

□  4.  Somewhat  easy 

□  5.  Very  easy 

2.  How  would  you  change  the  control  mechanism  to  improve  crosshair  positioning? 
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SECTION  3:  EVALUATION  OF  PART-TASK  SIMULATION 

This  section  contains  questions  about  the  realism  of  the  part-task  simulation.  We  are 
interested  in  your  comments  regarding  quality  of  the  simulated  SAR  imagery,  the 
procedure  used  to  accomplish  the  radar  updates,  the  mechanization  of  the  cursor, 
and  the  quality  of  the  RFP  cards. 

A.  Realism/Quality  of  SAR  Imagery 

1 .  Overall,  how  would  yot  rate  the  realism  of  the  simulated  SAR  imagery  presented 
during  the  demonstration? 

□  1 .  Very  unrealistic 

□  2.  Somewhat  realistic 

□  3.  Very  realistic 

□  4.  Don’t  know 


2.  What  simulated  SAR  effects/characteristics  would  you  improve? 


3.  Do  you  feel  the  types  (tanks,  bridges,  buildings,  towers,  etc.)  selected  for  the 
demonstration  are  typical  of  those  you  experience  in  your  operational  units? 


□  Yes 

□  No 

□  Don’t  know 


3.a.  If  not,  why? 


4.  As  you  know,  the  levels  of  CONTRAST  and  BRIGHTNESS  were  preset  for  the 
simulated  SAR  imagery.  Rate  the  selected  settings  in  terms  of  desirability. 

□  l.  Poor 

□  2.  Average 

□  3.  Good 


Explain: 


B.  Radar  Update  Procedure 


1 .  How  effective  was  the  demonstration  in  simulating  the  procedure  used  for 
performing  radar  updates? 

□  1 .  Very  ineffective 

□  2.  Somewhat  ineffective 

□  3.  Average 

□  4.  Somewhat  effective 

□  5.  Very  effective 

□  6.  Don’t  know 


Explain: 


C.  Mechanization  of  the  Cursors 


1 .  How  effective  was  the  demonstration  in  simulating  the  mechanization  of  the 
cursor? 

□  1 .  Very  ineffective 

□  2.  Somewhat  ineffective 

□  3.  Average 

□  4.  Somewhat  effective 

□  5.  Very  effective 

□  6.  Don’t  know 


Explain: 


I  ;Tn  I  [hiM'j  HU  il  d  w* 


1.  Rate  the  realism  of  the  RFP  cards  used  during  the  demonstration. 

□  l.  Very  unrealistic 

□  2.  Somewhat  realistic 

□  3.  Very  realistic 

□  4.  Don't  know 


Explain: 
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APPENDIX  C 

Questionnaire  and  Debrief  Results 
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SECTION  1:  BIOGRAPHICAL  DATA 
Summary  of  Biographical  Data  for  18  Subjects 


AIRCRAFT 

TYPE 

POSITION 

SENSOR 

EXPERIENCE 

OPERATING 

CROSSHAIR 

B-2  =  2 

B-l  =8 

B-52  =  13 

•Other  =  6 

OSO  =  7 

WSO  =  2 

RN  =  8 

NAV  =  7 

Pilot  =  1 

EW  =  3 

SAR  =  9 

EVS/FLIR  =  14 

17 

•Other  aircraft  types  included  F-15,  F-ll,  F-4,  F-16,  and/or  KC-135 
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SECTION  2:  EVALUATION  OF  THE  CURSOR 


2A.  Cursor  Type 

2A-1.  In  general,  visually  locating  or  seeing  the  cursor. 


Visually  Locating  the 
Cursor 


£■ 


o  Cursor  1  Cursor  2  Cursor  3 

mean  2.78  3-89  4.06 

S.d  1.48  0-76  0.64 


2A-1.  Subjects'  Comments 

Subject  #  Comments 

51  -  Cursor  1  was  easier  to  see  than  Cursors  2  and  3. 

52  -  Cursor  1  was  obscured  in  the  light  (bright)  return,  and  it  made  it  difficult  to 

see  exactly  where  the  center  of  the  cursor  was. 

53  -  Cursor  1  was  somewhat  difficult  because  it  got  buried  in  the  background. 

-  Cursor  2  was  the  easiest  because  of  the  perpendicular  end  points. 

54  -  Cursor  1  was  somewhat  difficult  when  it  appeared  over  bright  terrain. 
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55  -  Cursor  1  was  somewhat  difficult,  particularly  when  there  was  a  lot  of 

blooming.  The  cursor  got  lost  in  the  background  sometimes. 

56  -  Cursor  1  was  difficult  to  locate  in  a  cluttered  environment. 

-  The  perpendicular  end  points  on  Cursor  2  helped  me  find  it. 

S9  -  Cursor  3  was  the  easiest  to  find  because  of  end  bars  and  large  size. 

510  -  Cursor  1  got  lost  in  the  bright  or  light  returns.  Cursor  3  was  the  easiest. 

511  -  Cursor  1  was  easiest  to  locate;  others  were  somewhat  easy  to  locate. 

512  -  Cursor  1  was  more  difficult  because  of  the  width  of  the  bars  (i.e., 

brightness). 

-  Cursors  2  and  3  were  easier  because  of  the  long  bars.  They  don't  tend  to  get 

buried  in  bright  returns. 

513  -  Cursor  1  was  difficult  in  a  high  gain  or  large  return  environment.  I  found 

myself  moving  off  when  such  an  environment  existed.  Cursor  3  was  the 
easiest  to  see  because  of  the  large  end  points. 

514  -  Cursor  1  was  difficult  because  it  blends  in  with  high  returns.  Cursor  3  was 

the  easiest  to  pick  out  from  the  background. 

515  -  Cursor  1  was  difficult  because  it  looked  like  a  return. 

516  -  Cursor  1  was  easiest  to  see  because  it  was  so  bright. 

S18  -  Cursor  2  was  easiest  to  see  because  of  end  points. 

2A-2.  Accurately  visualizing  the  center  of  the  cursor  intersection. 


Accurately  Visualizing 
Center  of  Cursor 


Cursor  1  Cursor  2  Cursor  3 

mean  2.28  4.17  4.11 

s.d.  i.02  1  04  1.13 


2A-2.  Subjects'  Comments 

Subject  #  Comments 

51  -  Cursor  1  was  somewhat  difficult  because  the  bars  are  too  wide 
-  Cursors  2  and  3  were  easier  because  of  narrower  bars. 

52  -  Cursors  1  and  3  got  lost  in  the  white  areas  of  the  presentation. 

53  -  Cursor  1  was  difficult  because  of  the  fatness  of  the  bars. 


48 


54  -  It  was  very  easy  to  see  Cursors  2  and  3  because  the  lines  were  very  thin. 

-  Cursor  1  was  more  difficult  because  of  the  thickness  or  wideness  of  arms. 

55  -  Cursor  1  was  somewhat  difficult  because  of  the  thickness  of  the  bars. 

56  - 1  didn't  like  the  small  opening  of  Cursor  2. 

5 7  -  Cursor  1  was  more  difficult  because  of  the  width  of  die  bars  and  large 

center  opening. 

58  -  Cursor  1  was  difficult  because  of  its  large  opening. 

59  -  Cursor  1  was  difficult  because  of  its  brightness  and  boldness. 

510  -  Cursor  1  was  somewhat  difficult;  Cursors  2  and  3  were  both  easy. 

511  -  Cursor  1  was  more  difficult. 

512  - 1  believe  I'm  less  accurate  with  Cursor  1  because  of  the  wide  bars;  I  can't 

line  up  as  well. 

-  It  was  easier  to  visualize  the  center  with  Cursors  2  and  3  because  of  the  fine 

lines. 

513  -  Cursor  1  was  difficult  because  of  the  wideness  of  the  bars.  Cursors  2  and  3 

were  both  easy. 

514  -  Cursor  1  was  the  easiest  because  of  the  large  opening. 

515  -  Cursor  1  was  very  difficult  because  of  the  width  of  the  bars. 

516  - 1  liked  the  larger  opening  of  Cursor  1.  Cursor  3  was  very  easy  because  the 

end  bars  led  your  eye  into  the  center  intersection. 

517  -  Cursor  1  obscured  the  fix  point. 

518  -  Because  I  lined  up  one  axis  at  a  time.  Cursors  2  and  3  made  that  easy.  I 

couldn't  do  that  with  Cursor  1. 
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2A-3.  Accurately  positioning  the  center  of  the  cursor  over  the  desired  designation 
point 


Accurately  Positioning 
Canter  of  Cursor  Over 
Aimpoint 


Cursor  1  Cursor  2  Cursor  3 

mean  2.44  4.22  4.11 

s.d.  1.11  1.00  1.10 


2A-3.  Subjects'  Comments 


Subject  # 

SI 


52 

53 
S5 
S  7 

58 

59 

510 

511 

512 


513 

514 
S17 


Comments 

-  This  gets  into  the  problem  of  mechanization.  I  object  to  the  fact  that  I  can’t 

move  diagonally. 

-  Cursor  1  had  an  additional  step  to  move  off  the  area;  then  begin 

positioning. 

-  Cursor  2  was  the  definite  favorite  because  the  others  got  lost  in  white  areas 

of  return. 

-  All  cursors  were  difficult  because  of  the  crosshair  control. 

-  Cursor  2  was  the  easiest. 

-  All  cursors  were  downgraded  because  of  the  cursor  control.  Mechanization 

made  an  easy  task  difficult. 

-  Cursor  1  was  difficult  because  of  its  large  opening. 

-  Cursor  1  was  difficult  because  of  its  brightness  and  boldness. 

-  Cursor  1  was  somewhat  difficult;  Cursors  2  and  3  were  both  easy. 

-  Cursor  1  was  difficult  because  of  the  coarseness  of  the  bars. 

- 1  believe  I’m  less  accurate  with  Cursor  1  because  of  the  wide  bars;  I  can’t 
line  up  as  well. 

-  It  was  easier  to  visualize  the  center  with  Cursors  2  and  3  because  of  the  fine 

lines. 

-  Cursor  1  was  difficult  because  of  the  wideness  of  the  bars.  Cursors  2  and  3 

were  both  easy. 

- 1  didn't  like  the  small  opening  of  Cursors  2  and  3. 

- 1  felt  Cursor  1  degraded  accuracy.  I  felt  handicapped  because  of  the 
coarseness  of  the  design. 
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2A-4L  Quickly  positioning  the  center  of  the  cursor  over  i’te  desired  designation  point 


Quickly  Positioning  Center 
of  Cursor  Over  Aimpoint 


Cursor  1  Cursor  2  Cursor  3 

mean  3.11  3.72  3.72 

s.d.  1.23  0.83  0.96 


2A-4.  Subjects' Comments 
Subject  #  Comments 

51  -  Cursor  1  gross  and  fine  positioning  were  very  slow.  Cursors  2  and  3  gross 

positioning  was  fine;  but  fine  positioning  was  difficult  because  of 
mechanization.  Cursor  3  was  a  little  harder  because  of  the  wide  end  bars. 
End  bars  were  a  little  distracting  on  Cursor  3. 

52  -  Cursor  1  failed  miserably  in  all  areas. 

53  -  Cursor  1  was  difficult  because  it  obscured  what  was  in  the  background  and 

that  impacted  my  speed. 

S5  -  Cursor  2  was  easiest. 

S7  -  Cursors  2  and  3  were  easier  than  Cursor  1  because  of  narrower  bars. 

S9  -  Cursor  1  was  difficult  because  of  its  brightness  and  boldness. 

512  -  Cursor  1  seems  the  fastest,  but  I  question  my  accuracy. 

513  -  The  length  of  Cursors  2  and  3  helped  me  in  terms  of  speed. 

514  -  Cursor  1  was  easiest  because  of  the  large  opening. 

515  -  Cursor  1  was  the  most  difficult  because  I  always  had  to  move  off  to  see 

what  was  underneath.  Cursor  3  was  the  easiest. 

S18  -  Cursor  1  had  to  be  moved  out  of  the  way  before  starting  to  position;  wasted 

time. 
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2A-5.  Designating  aim  points  in  a  high-clutter  environment 


Designating  Aimpoints  in 
High-Clutter  Environment 


Cursor  1  Cursor  2  Cursor  3 


mean  1.83  3.44  3.38 

s.d.  1.04  1.04  0.02 


2A-5.  Subjects' Comments 


Subject  # 

SI 


54 

55 

56 

5 7 

58 


S10 

512 

513 

514 

515 

516 

517 


Comments 

-  Cursor  1  was  difficult  in  a  cluttered  environment  because  it  obscured 

everything  in  the  area  of  interest. 

-  Cursor  2  long  bars  helped  me  line  up  past  bright  returns  in  the  area  of 

interest.  I  would  like  to  see  a  larger  opening  on  Cursors  2  and  3. 

-  Brightness  of  Cursor  1  makes  it  difficult  in  images  with  bright  return. 

-  Length  of  Cursors  2  and  3  made  it  easier. 

-  Cursor  1  was  difficult  because  it  obscured  the  area  of  interest. 

-  Because  of  the  width  of  Cursor  1,  it  appeared  brighter  than  many  of  the 

returns.  It  also  obscured  things. 

-  Cursor  1  was  somewhat  difficult  because  of  the  width  and  shortness  of  the 

bars. 

-  Because  of  high  reflectivity.  Cursor  1  was  blanked  out  at  times. 

-  Cursors  2  and  3  are  a  little  easier  because  of  the  length  of  the  cursors,  but  it 

is  still  difficult  to  center  over  the  exact  "golden  pixel". 

-  Cursor  1  was  difficult  because  of  the  bright  video  surrounding  it.  Cursor  3 

was  the  best  in  high  clutter. 

-  When  it's  bright  you  can’t  find  Cursor  1.  Very  difficult. 

-  Cursor  1  was  very  difficult  in  high  clutter.  Cursor  3  was  easier  than  Cursor 

2  because  of  end  points. 

-  Cursor  1  was  easiest  because  of  its  large  opening. 

-  Cursor  1  was  the  most  difficult  because  it  blended  in  with  returns. 

-  Cursor  3  was  the  easiest  because  of  the  end  points. 

-  Cursor  1  was  very  difficult  because  it  washed  out  when  near  or  over  other 

returns. 
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S18  -  Cursor  1  was  more  difficult  than  the  others. 

2A-6.  Designating  aim  points  in  an  uncluttered  environment 


Designating  Aimpoints  in 
Uncluttered  Environment 


Cursor  1 


mean  3.61 
s.d.  1.24 


Cursor  2  Cursor  3 

4.61  4.44 

0.61  0.70 


2A-6.  Subjects' Comments 
Subject#  Comments 

S4  -  The  lengths  of  Cursors  2  and  3  made  it  very  easy  to  designate  in 

uncluttered  environments. 

S9  -  All  cursors  were  easy. 

511  -  All  cursors  were  easy. 

512  -  All  the  cursors  were  easy  when  there  were  no  bright  returns. 

513  -  Cursor  1  was  most  difficult  because  of  the  wideness  of  bars.  I  liked  Cursor 

2  in  an  uncluttered  environment. 

514  -  All  cursors  performed  the  same  in  uncluttered  environment. 

S16  -  All  cursors  were  easy. 

S18  -  All  cursors  were  easy. 
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2A-7.  Please  rate  the  overall  effectiveness  of  each  cursor  type  for  designating  aim 
points. 


Overall  Effectiveness 


Cursor  1  Cursor  2  Cursor  3 

mean  2.17  4.56  4.50 

s.d.  0.91  0.94  1.04 


2A-7.  Subjects' Comments 


Subject# 

SI 


S2 

S4 


55 

56 

5 8 

59 

510 

511 


512 

513 

515 

516 

517 

518 


Comments 

-  Cursor  1  was  ineffective  because  of  its  overall  size  and  width  of  the  bars. 

-  Cursor  2  was  most  effective  because  of  its  overall  size,  narrow  bars  and  end 

points. 

-  Cursor  2  was  the  clear  favorite  for  me. 

-  Cursor  2  was  very  effective.  I  didn’t  like  Cursor  3  as  well,  but  I  am  not  sure 

why. 

-  Cursor  1  was  ineffective  because  of  shortness  and  thickness  of  bars. 

-  Cursor  1  was  the  worst;  Cursor  2  was  the  best. 

-  Cursor  1  was  most  difficult  to  find.  It  covered  things  up. 

-  Cursors  2  and  3  were  easier  in  terms  of  designating  accurately. 

-  Cursor  3  was  the  best  because  of  the  end  points. 

-  Cursor  1  was  very  ineffective;  Cursor  3  is  very  effective  because  of  the 

longer  legs  that  lead  into  the  center. 

-  Cursor  3  was  the  most  effective  because  of  end  points;  these  made  it  easy  to 

find  it  and  move  it  into  the  area  of  the  aim  point.  I  didn't  like  the  narrow 
end  points  on  Cursor  2. 

-  Cursor  1  was  somewhat  effective. 

-  Cursor  3  was  most  effective. 

-  Cursor  1  was  ineffective;  I  thought  both  Cursors  2  and  3  were  very 

effective. 

-  Cursor  3  was  the  most  effective. 

-  Cursor  1  was  very  ineffective  because  I  felt  my  accuracy  was  compromised. 

-  Cursors  2  and  3  were  very  effective. 


54 


2A-8.  Which  cursor  provided  the  best  overall  performance? 


Best  Overall  Performance 


2A-8a.  Why? 

Subject  #  Comments 

51  - 1  liked  Cursor  2  the  best  because  of  its  narrow  arms;  the  terminators  helped 

me  find  the  cursor  and  line  up  for  designating.  I  also  thought  the 
terminators  on  Cursor  3  were  a  little  too  wide. 

52  -  Cursor  2  was  the  easiest  to  position  over  the  target.  I  liked  the  larger  size 

for  lining  up. 

54  -  Cursor  2  performed  the  best  because  of  the  vertical  bars  at  the  end.  Cursor 

3  end  bars  might  obscure  in  some  situations. 

55  -  Cursor  2  was  the  easiest  to  locate  because  of  vertical  terminators.  I  had  an 

easier  time  visualizing  the  intersection  of  cursor. 

56  - 1  liked  Cursor  2  because  of  the  perpendicular  terminators  and  small  width 

of  crosshairs.  I  would  like  to  see  the  cursor  even  bigger,  like  on  the  B-l. 

5 7  - 1  liked  Cursor  2  because  of  the  light  contrast  between  it  and  surrounding 

display;  large  enough  size;  and  thinness  of  the  bars. 

58  - 1  liked  Cursor  2  because  I  could  designate  faster .  It  also  provided  the  most 

detail  and  provided  the  best  accuracy. 

59  - 1  liked  Cursor  3  because  of  the  thinness  of  the  inside  and  wideness  of  the 

outside  end  points. 

510  - 1  liked  Cursor  3  because  of  the  longer  arms  that  lead  into  the  center. 

511  - 1  liked  Cursor  3  the  best  because  of  the  end  points  and  large  size. 

512  -  The  bars  on  the  ends  of  Cursor  2  made  it  slightly  easier  for  me  to  find  and 

to  aim. 

-  I  liked  how  small  the  center  opening  is  on  Cursors  2  and  3. 
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S13  - 1  liked  Cursor  3  because  the  fatter  end  points  helped  over  a  wide  range  of 

gain  and  contrast  settings.  The  smaller  opening  is  an  improvement.  Also, 
the  longer  and  narrower  lines  pointing  to  the  center  was  a  big  help. 

515  - 1  liked  Cursor  2  because  of  perpendicular  end  points. 

516  - 1  liked  Cursor  3  because  of  its  outer  border;  it  was  good  for  gross 

positioning  and  the  thinner  bars  were  good  for  fine  positioning. 

51 7  - 1  liked  Cursor  3  because  it  was  easy  to  acquire,  contrast  in  the  high  and  low 

clutter;  I  used  the  fat  part  to  initially  acquire,  then  the  fine  lines  for  fine 
positioning.  In  my  opinion.  Cursor  3  provides  the  best  of  both  worlds. 

518  - 1  liked  Cursor  3  because  of  the  longer  bars. 

2A-9.  How  would  you  improve  upon  the  cursor  listed  in  question  8  (the  best 
performing  cursor)  and  why  do  you  consider  this  an  improvement? 


Subject  # 

51 

52 

53 

54 

55 

56 

57 


S8 


S10 

514 

515 

516 

517 


Comments 

-  Increasing  the  size  of  the  opening  would  be  an  improvement  on  Cursor  2. 

-  Have  cursor  a  different  color  than  the  screen. 

-  A  dashed  line  for  Cursor  2,  which  will  get  smaller  as  you  get  to  the  center. 

-  From  my  B-1B  experience,  make  the  cursor  longer. 

-  Cursor  2  is  already  very  effective.  No  suggestions. 

-  Cursor  2  makes  the  center  larger.  Needs  a  capability  to  time  intensity  of 

cursor. 

-  Improvement  to  Cursor  2:  It's  a  little  large  to  take  advantage  of  cardinal 

effect,  and  as  cross  reference  to  larger  field  of  view  (i.e.,  surrounding  aim 
points). 

-  Improvement  to  Cursor  2:  Allow  for  changing  the  color  of  the  cursor  in 

order  to  provide  contrast  between  cursor  and  background  image.  Smart 
cursor  or  auto /manual  contrast. 

-  Improvement  to  Cursor  3:  Make  the  end  points  blink;  this  would  be  helpful 

in  a  stressful,  bumpy  environment. 

-  Improvement  to  Cursor  1:  Change  the  color  or  use  multiple  sizes  of 

cursors. 

-  Improvement  to  Cursor  2:  Make  the  crosshairs  even  longer. 

-  Improvement  to  Cursor  3:  Make  the  center  opening  larger. 

-  Improvement  to  Cursor  3:  Capability  to  select  reverse  video  or  flashing 

cursor  so  I  can  maximize  the  contrast  between  cursor  and  background. 
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2B.  Designation  Technique 


2B-1.  Describe  the  overall  technique  you  used  for  crosshair  positioning  and 
designating  aim  points  during  the  demonstration. 


Subject  # 

51 

52 

53 

54 

55 

56 

57 

58 

59 

511 

S10 

512 
SI  3 

514 

515 
Si  7 
S18 


Comments 

-  Cursor  1  required  an  extra  step  in  that  you  had  to  move  the  cursor  off  the 

image  m  order  to  see  what  was  underneath.  Other  than  that,  I  used  the 
same  strategy  for  all  cursors. 

- 1  used  the  same  strategy  for  Cursors  1  and  2.  Lined  up  one  axis  at  a  time  for 
Cursor  3. 

- 1  used  the  same  technique  for  Cursors  2  and  3.  Had  to  move  cursor  off  with 
Cursor  1. 

- 1  had  to  move  Cursor  1  off  the  area  of  interest.  Cursor  1  obscured  aim 
points. 

- 1  moved  Cursor  1  off  in  order  to  see  area  of  interest.  Moved  one  axis  at  a 
time.  I  used  direct  placement  for  Cursors  2  and  3. 

- 1  had  to  move  Cursor  1  out  of  the  way  initially,  but  didn't  have  to  move 
Cursors  2  and  3.  Other  than  that  I  used  the  same  technique. 

- 1  used  an  additional  step  for  Cursor  1,  which  was  to  move  the  cursor  off  in 
order  to  see  area  of  interest. 

- 1  used  the  same  technique  for  Cursors  2  and  3:  one  axis  at  a  time.  Used 
ends  of  cursors  to  line  up. 

- 1  used  the  same  technique  for  all  three  cursors. 

- 1  used  the  same  technique  for  all  but  I  felt  I  had  the  ability  to  perform  fine 
positioning  with  Cursors  2  and  3. 

-  Cursor  1  required  that  I  had  to  guess  where  the  golden  pixel  was 

positioned.  I  used  the  same  technique  with  Cursors  2  and  3,  but  felt  I  was 
more  accurate  because  of  the  small  opening. 

- 1  used  thin  bars  of  Cursors  2  and  3  to  line  up  one  axis  at  a  time. 

-  Cursor  1  was  pulled  off  initially  to  see  the  aim  point.  It  was  particularly 

difficult  to  use  in  high  gain  settings.  I  found  with  all  cursors  that  I  settled 
for  being  "close  enough"  because  of  mechanization.  Cursors  2  or  3  were 
able  to  be  moved  right  to  the  aim  point,  then  lined  up  one  axis  at  a  time. 

-  Cursor  1  required  an  extra  step  of  moving  off  the  aim  point  to  see  what  was 

underneath. 

-  There  was  an  extra  step  with  Cursor  1  of  pulling  the  cursor  off  to  see 

underneath.  For  Cursors  2  and  3 1  lined  up  one  axis  at  a  time. 

-  Cursor  1  required  an  extra  step  in  that  you  had  to  move  the  cursor  off  the 

image  in  order  to  see  what  was  underneath. 

-  Cursor  1  required  an  extra  step  in  that  you  had  to  move  the  cursor  off  the 

image  in  order  to  see  what  was  underneath.  Other  than  that  I  used  the 
same  strategy  for  all  cursors. 
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2C.  Mechanization 


201.  In  terms  of  designation  speed  and  accuracy,  how  would  you  rate  the  difficulty 
of  controlling  the  cursor  position? 


Mechanization 


Cursors  1,2,  and  3 

mean  1.72 

sd  o.96 _ 


202.  How  would  you  change  the  control  mechanism  to  improve  crosshair 
positioning? 

Subject#  Comments 

SI  - 1  could  not  move  diagonally.  Gross  movement  is  fine,  particularly  with 

Cursors  2  and  3.  Fine  positioning  is  very  difficult.  Mechanization  causes 
the  cursor  to  jump  past  the  desired  designation  point.  Also,  with  Cursor  1 
there  is  an  additional  motion  in  that  you  have  to  move  the  cursor  off  in 
order  to  see  what  it  is  covering. 

51  -  Would  improve  by:  (1)  decreasing  deadband,  (2)  removing  residual 

motion,  (3)  improving  diagonal  control,  and  (4)  modifying  slew  curve  for 
fine  positioning. 

52  - 1  had  trouble  moving  one  pixel  at  a  time.  I  would  like  to  be  able  to  lock 

position  in  on  one  axis  at  a  time. 

53  -  It  was  okay  for  gross  positioning  but  awful  for  fine  positioning.  I  would 

have  two  separate  modes.  Fine  positioning  would  be  a  slower  movement 
(maybe  use  a  trackball). 

54  - 1  had  difficulty  performing  fine  positioning  for  all  cursors.  I  had  to  bump 

and  then  did  a  lot  of  overshooting.  I  would  recommend  two 
mechanization  rates;  one  for  gross  and  one  for  fine  positioning.  It  would 
then  allow  for  fast  movement  and  slow  movement  for  precise  positioning. 

55  -  Need  a  two  rate  mechanization;  fast  and  slow  rates.  I  also  think  a  trackball 

is  easier  to  use  than  the  slew  button. 

56  - 1  had  to  put  too  much  pressure  on  thumb  position.  Something  needs  to  be 

accomplished  in  order  to  eliminate  overshooting.  I  didn't  like  the 
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hardware;  pull  trigger  to  activate,  then  release  to  designate.  I  didn't  like 
the  slew  curve 

57  -  It  was  okay  for  gross  positioning;  very  difficult  for  fine  positioning.  I  didn't 

like  the  fact  that  I  couldn't  move  it  diagonally.  This  kept  me  from 
designating  the  pixel  I  wanted  and  also  impeded  accuracy. 

58  -  Mechanization  very  much  like  the  aircraft  -  very  difficult.  It  takes  too  much 

pressure  to  move  the  cursor.  Need  to  readjust  slew  rate  to  make  the  fine 
positioning  easier. 

59  -  Stiff  controls  made  it  difficult  for  fine  positioning  (i.e.,  cursor  jumped  past 

where  I  wanted  to  be).  Increase  the  amount  that  you  can  move  the  button; 
also  use  a  variable  rate. 

510  -  Control  would  go  past  where  I  wanted  to  go.  Couldn't  move  it  one  pixel  at 

a  time.  In  a  low  level,  turbulent  environment  it  would  be  very  difficult.  I 
would  like  constant  velocity.  Cursor  should  stop  immediately,  but  it 
seemed  to  keep  running  past  where  I  wanted  to  go. 

511  -  Completely  unsatisfactory.  Movement  was  not  proportional  to  force 

applied.  An  improvement  would  include  decreasing  the  deadband.  I 
would  like  the  control  itself  to  move  more  to  provide  better  feedback. 

51 2  -  Mechanization  is  awful.  I  didn't  like  the  thumb  controller.  The  fine 

positioning  is  very  difficult.  Suggest  looking  at  the  hardware  itself,  the 
slew  rate  curve  (lower  the  slope  on  the  rate),  and  the  deadband  (too  big?). 

513  -  Mechanization  was  very  difficult,  particularly  in  fine  positioning.  There  is 

a  lot  of  bumping.  Cursors  2  and  3  helped  because  of  narrow  lines. 
Changes:  didn't  seem  to  act  the  same  each  time  with  similar  pressure  (was 
too  jerky).  I  couldn't  find  the  sensitive  spot  (always  tapping). 

514  -  It  was  difficult  because  I  felt  the  rate  was  unpredictable.  Changes  would 

include  operator  being  able  to  change  the  slew  rate. 

515  -  It  was  very  difficult  because  of  the  jumping  of  the  cursor.  It  would  be 

particularly  difficult  in  a  turbulent  environment.  It  didn't  seem  to  move 
one  pixel  at  a  time.  I  would  like  to  see  greater  movement  on  the  slew 
button  to  provide  feedback. 

516  - 1  felt  the  cursor  was  responsive  but  also  would  jump  past  where  I  wanted  to 

be  at  times.  Possibly  change  to  a  thumb  trackball  or  something  that  is 
easier  to  move  back  and  forth. 

517  -  The  mechanization  is  very  difficult.  This  is  the  number  one  thing  to  change. 

I  would  want  to  be  able  to  move  the  cursor  1-2  pixels  at  a  time.  Awful  for 
fine  positioning. 
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SECTION  3:  EVALUATION  OF  PART-TASK  SIMULATION 


3A.  Realism/Quality  of  SAR  Imagery 

3A-1.  Overall,  how  would  you  rate  the  realism  of  the  simulated  SAR  imagery 
presented  during  the  demonstration? 


NUMBER 

OF 

RESPONDENTS 

MSISM 

MEAN 

STANDARD 

DEVIATION 

17 

3 

2.67 

.61 

3A-2.  What  simulated  SAR  effects/characteristics  would  you  improve? 


Subject#  Comments 

55  -  Overall,  the  best  I’ve  seen  in  a  simulator. 

56  -  Shadowing  on  peaks  could  have  been  improved. 

511  -  Pixels  were  too  large  and  too  grainy.  Shadowing  was  unrealistic. 

512  -  Good  job  on  imagery. 

S15  -  Didn't  seem  to  be  an  adequate  amount  of  shadowing  of  buildings  and  silos. 

3A-3.  Do  you  feel  the  types  (tanks,  bridges,  buildings,  towers,  etc.)  selected  for  the 
demonstration  are  typical  of  those  you  experience  in  your  operational  units? 


NUMBER 

OF 

RESPONDENTS 

POSSIBLE 

RATINGS 

RESPONSES 

17 

YES/NO/DONT 

KNOW 

100%  =  Yes 

3A-3a.  If  not,  why  not? 


Subject# 

SI 

58 

59 

511 

512 
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Comments 

-  "Southwest  end  of  ridge  line"  and  "end  of  butte"  are  not  realistic  aim 

points  for  SAR. 

-  Might  want  to  include  some  intersections  of  runways. 

- 1  wouldn't  use  terrain  features  for  SAR  (top  of  the  mountain,  butte). 

-  Good  selection  of  aim  points. 

-  Terrain  is  not  good  for  SAR. 


3A-4.  As  you  know,  the  levels  of  CONTRAST  and  BRIGHTNESS  were  preset  for  the 
simulated  SAR  imagery.  Rate  the  selected  settings  in  terms  of  desirability. 


NUMBER 

OF 

RESPONDENTS 

MAXIMUM 

DESIRABILITY 

RATING 

MEAN 

STANDARD 

DEVIATION 

18 

3 

2.44 

.62 

3A-4.  Subjects'  Comments 


Subject#  Comments 

55  -  Need  to  be  able  to  adjust  to  reduce  the  blooming. 

56  -  Tune  down,  especially  with  no  cursor  contrast  control. 

S8  -  Edges  of  ridges  imagery  was  too  bright  and  I  needed  to  tone  them  down. 

Terrain  returns  needed  the  most  work. 

512  -  An  overall  good  job,  although  I  didn’t  like  the  fact  that  it  was  fixed. 

513  -  Lower  gain  setting  and  brightness  would  have  helped  to  ID  the  cursor 

earlier. 

3B.  Radar  Update  Procedure 

3B-1.  How  effective  was  the  demonstration  in  simulating  the  procedure  used  for 
performing  radar  updates? 


NUMBER 

OF 

RESPONDENTS 

MAXIMUM 

EFFECTIVENESS 

RATING 

MEAN 

STANDARD 

DEVIATION 

16 

5 

4.44 

1.03 

3B-1.  Subjects'  Comments 

Subject#  Comments 

S 8  -  Procedure  didn't  include  all  the  procedural  stuff  that  the  operator  can  do  to 

perform  a  good  update,  (changing  patch  sizes,  adjusting  brightness  and 
contrast  controls) 

S15  - 1  didn't  like  the  time  pressure.  Never  seemed  to  be  time  pressure  in 

operational  environment  (B-l). 
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3G  Mechanization  of  the  Cursors 


3C-1.  How  effective  was  the  demonstration  in  simulating  the  mechanization  of  the 
cursor? 


NUMBER 

OF 

RESPONDENTS 

MAXIMUM 

EFFECTIVENESS 

RATING 

MEAN 

STANDARD 

DEVIATION 

8 

5 

4.5 

1.19 

3C-1.  Subjects' Comments 

Subj  ect  #  Comments 

S8  -  This  shows  how  awful  the  mechanization  is  in  the  actual  aircraft. 


3D.  Realism  of  the  RFP  Cards 


3D-1.  Rate  the  realism  of  the  RFP  cards  used  during  the  demonstration. 


NUMBER 

OF 

RESPONDENTS 

MAXIMUM 
REALISM  RATING 

MEAN 

STANDARD 

DEVIATION 

15 

3 

2.60 

.51 

3D-1.  Subjects'  Comments 


Subject#  Comments 

S8  -  Cards  were  fuzzier  than  the  actual  cards. 

Sll  -  The  description  of  aim  points  could 'have  been  improved  and  the  quality  of 
the  photographs  was  dismal. 

S15  - 1  didn't  like  the  digitized  image. 
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